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Executive Summary 


Headquartered in Washington, DC, Urban Alliance serves at-risk youth through its High School 

Internship Program, which provides training, mentoring, and work experience to high school seniors 
from distressed communities in Washington, DC; Baltimore, MD; Northern Virginia; and Chicago, IL. 
The program serves youth before they become disconnected from school and work, with the goal of 


helping them successfully transition to higher education or employment after graduation. 


Urban Alliance commissioned the Urban Institute to conduct a six-year, randomized controlled trial 
impact and process evaluation of its High School Internship Program. A first report (Theodos et al. 
2014) provided a process analysis of the program and baseline information about Urban Alliance and 
the youth participating in its High School Internship Program in Washington, DC, and Baltimore in the 
2011-12 and 2012-13 program years. A second report (Theodos, Pergamit, Hanson, et al. 2016) shared 


interim impact findings. This report describes final impact findings. 


What Is the Urban Alliance Program Model? 


= Goal: The Urban Alliance High School Internship Program strives to change the trajectories of 
youth who are at risk of becoming disconnected, neither attending college nor finding stable 
employment. It intervenes in their lives at a critical juncture—their senior year of high school— 


and offers them training, an internship, and mentoring to help them succeed. 


= Targeting: Urban Alliance targets seniors in high school at risk of not transitioning to further 
education or meaningful work. It aims to serve middle-of-the-road high school students who 


maintain a grade point average (GPA) of 2.0 to 3.0, but it is not limited to that group. 


= Program components: The program’s key elements are (1) a paid internship in an office setting, 
(2) soft and hard skills job training, which occurs both before the internship in the “pre-work” 
phase as well as through the remainder of the year, concurrent with the internship, (3) coaching 
and mentoring provided by Urban Alliance program coordinators and job mentors at the 
internship site, and (4) alumni services consisting of individual coaching, alumni events, and paid 


internship opportunities during the summer break from college. 
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What Was the Study Design? 


Research Questions 


Do youth who participate in the program exhibit stronger hard and soft skills than a 
control group of youth not enrolled in the program? 

Does the Urban Alliance program lead to increased rates of college enrollment and 
persistence for participants compared with a control group of youth not enrolled in the 
program? 

Do Urban Alliance participants have higher rates of employment and earnings than 


control group youth? 


Random assignment: We assigned applicants from program years 2011-12 and 2012-13 in 


Washington, DC, and Baltimore at random to a treatment or control group. We randomized 


applicants separately at each site by using a two-to-one ratio of treatment to control. Urban 


Alliance invited only those applicants assigned to the treatment group to participate in the 


program. 


Data collection: We collected quantitative data from various sources: 


baseline demographic and education data from the Urban Alliance application form 
baseline neighborhood characteristics from the American Community Survey, 2008- 
2012 

high school transcript data 

program participation data from Urban Alliance 

survey data from two follow-up surveys at roughly one year and three months (referred 
to as the one-year mark) and roughly two years and three months (referred to as the two- 
year mark) on both treatment and control youths’ services receipt, skill development, 
education, and employment 

college enrollment and persistence data from the National Student Clearinghouse 
college quality data from the National Center for Education Statistics’ Integrated 


Postsecondary Education Data System 


Analysis methods: The randomized controlled trial approach allowed us to estimate the causal 


impacts of the Urban Alliance program on skills, college enrollment and persistence, and 


employment and earnings. Further details on our analytic approach are as follows. 
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Predicting treatment take-up: We used regression analysis to estimate how much 
individual and neighborhood characteristics related to the probability of attending pre- 
work, completing pre-work, and completing the internship. 

Intent-to-treat analysis: We used regression analysis to estimate the impact of the Urban 
Alliance program for those youth randomized into treatment compared with those 
randomized into the control group. In addition, we estimated separate regressions for 
each GPA group (low, middle, high), each site, and each gender. 

Treatment-on-the-treated analysis: We estimated the effects of completing the program by 
estimating the differences in each outcome between the treated (youth who completed 
the program) and control subjects. To correct for the fact that persistence in the program 
varied by characteristics we were unable to measure, we estimated the treatment-on- 


the-treated effects by using an instrumental variables approach. 


Who Were the Youth in the Study? 


= The study sample included 1,062 youth who applied to the Urban Alliance program and agreed 


to participate in the evaluation. The following is an overview of their baseline characteristics. 


» 


Demographic characteristics: Eighty-nine percent of applicants were non-Hispanic African 
American (“African American” for shorthand), and 65 percent of applicants were female. 
Over half the applicants lived only with a single parent, and 12 percent lived with neither 
parent. 

Work experience: Three-quarters of youth reported at least some work experience before 
applying for the program, with average experience of just less than 10 months in all jobs 
combined. 

Educational background: Most youth were middle-of-the-road students, with an average 
junior year cumulative GPA of 2.7. Slightly more than a quarter of Urban Alliance 
applicants attended a charter school, with the majority in Washington, DC. Over one- 
third of applicants had attended more than one high school. 

Neighborhood characteristics: Applicants typically resided in economically distressed 
neighborhoods. Nearly half lived in neighborhoods with poverty rates higher than 25 
percent. Almost all applicants resided in census tracts that were composed of over 75 


percent people of color. 
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» — School characteristics: Almost all schools that Urban Alliance applicants attended were 
majority African American, though some schools in DC also had significant Hispanic 
student contingents. About 93 percent of youth attended schools with the majority of 
students eligible for free or reduced-price school lunch. Forty-five percent of Urban 
Alliance applicants attended a school ranking in the bottom quartile of proficiency in 


reading and math in DC or Maryland. 


= Differences between treatment and control groups: There were very few differences across the 
treatment and control groups at baseline, indicating that randomization was successful for this 
study. However, a slightly higher proportion of the control group (97 versus 95 percent) 
reported being US citizens and had a banking account (42 versus 35 percent). In addition, the 
treatment group on average had a somewhat higher GPA than the control group (2.7 versus 


2.6) at the end of their junior years. 


What Services Did Youth Receive? 


The Urban Alliance model allows youth to self-select into the program and expects, by design, varying 
levels of attrition during pre-work (though this depends on the site). A student who applied and was 
assigned to the treatment group could self-select out of the program in three ways: by not showing up 


to pre-work training, not completing the pre-work training, or not completing his or her internship. 


= Program attrition: There is substantial attrition in the Urban Alliance internship program, 
primarily in the first two stages, before and during pre-work training. Of those students 
assigned to the treatment group, 22 percent did not attend any pre-work sessions. Of all 
treatment group youth, one-quarter (25 percent) began but did not complete pre-work 
training. The remaining 49 percent were placed in a job, and most of those (84 percent) 


completed the program. In all, 41 percent of treatment group youth completed the program. 


=" Take-up regressions: We estimated predictive models that related baseline characteristics of the 
youth and program to the likelihood that youth would complete each of the program stages. 
Overall the biggest predictor of completion was GPA, with the probability of completing an 
internship 19 to 23 percentage points higher for those students with GPAs of 2.0 to 4.0 than for 
those with GPAs below 2.0. Additionally, youth from the 2011-12 cohort were 14 percentage 
points more likely to complete the program than youth from the 2012-13 cohort. Our process 


evaluation revealed many reasons for youth exiting the program. Youth principally cited 
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competing priorities, such as athletics or extracurricular activities, class schedules, lack of 
interest in the training, family or personal issues, relocation, and cost of transportation as 


reasons for attrition. 


= Services received: Receiving college and job help was prevalent among youth in the control 
group, more than 80 percent reported accessing college or job help. We found that the 
difference between the treatment and control group of reported college help and job help 
receipt was statistically significant, but relatively small, at 8 and 13 percentage points, 


respectively. 


What Impacts Did the Program Generate? 


= College readiness and high school achievement: At the one-year mark, we examined the impacts of 
Urban Alliance on college preparation and high school achievement on the full group and on 


subgroups. 


» College readiness: For the full group, we found that Urban Alliance had an impact on 
youths’ self-reported comfort with filling out the Free Application for Federal Student 
Aid and applying for other scholarships. We did not find any statistically significant 
impacts on other measures of college preparation such as taking the SAT, taking the 
ACT,1 or whether the youth applied to college. We found similar results for the various 
subgroups, except for males, who were more likely to apply to college. 

» High school achievement: We did not find any impacts on any measure of high school 
achievement for the full group, but we found some impacts for males and students with 
a GPA of 2.0 to 3.0. The program increased the probability of graduating from high 
school for males in the treatment group compared to males in the control group. For 
the GPA 2.0 to 3.0 group, we found that those students who completed the program 
were more likely to be chronically absent senior year compared to students in the 


control group. 


= Skill development: We estimated the impacts of Urban Alliance on the persistence of hard and 
soft skill comfort. (Hard skills include things like faxing, Microsoft Excel basics, making 
photocopies, and filing; soft skills include things like speaking with adult coworkers, writing 
professional e-mails, making presentations, dressing professionally, completing work 


assignments on time and getting to work on time.) We found positive and significant impacts at 
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the one-year mark, but these effects faded by the two-year mark as the control group caught 


up. These findings were true for most subgroups except for males. 


Soft skill comfort: The program had positive and significant impacts on youth comfort 
with soft skills both one and two years after expected graduation; however, the size of 
the effect had fallen at the two-year mark. For the male subgroup, the program had a 
strong and positive impact on the comfort with soft skills at the one-year mark, and the 
effect grew by the two-year mark. 

Hard skill comfort: We found positive and significant impacts at the one-year mark, but 
this effect did not persist into the second year, which appears to be the result of the 
control group “catching up.” 

Goal setting and time management: We found no significant effect of the program on the 


measures of goal setting and time management. 


Educational attainment: We explored the extent to which Urban Alliance prepares youth for 


postsecondary education and employment by assessing measures of college attendance, the 


quality of colleges attended, and persistence at college. We did not find program impacts on 


attendance and persistence for the full sample and minimal effects on proxies for quality. 


However, we found impacts on college attendance and persistence for certain subgroups, 


particularly for males. 


College attendance: For the full sample, we did not detect impacts on college attendance 
or persistence measures. Although Urban Alliance had a positive and significant impact 
on the 75th percentile SAT score at the one-year mark, these results did not endure at 
the two-year mark. 

College for males: The program demonstrated large impacts for males on the probability 
of attending college and the probability of attending a four-year college at the one- and 
two-year marks. The Urban Alliance program increased the probability of college 
attendance by 12 percentage points for males offered the program and 23 percentage 
points for males who completed the program. Similarly, the program increased the 
probability of attaining a two-year degree or being enrolled in their third year by 10 
percentage points for males offered the program and by 21 percentage points for 
males who completed the program. These results were driven by males in the 
treatment group attending four-year colleges rather than two-year colleges. 

College for students with differing GPAs: Urban Alliance had no significant impact on 


college attendance for the low-GPA subgroup; college enrollment was low for that 
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group. For the middle-GPA subgroup, youth offered treatment and those completing 
the internship were both more likely to attend a four-year college than the control 
group (by 9 and 17 percentage points, respectively, at the two-year mark). For the 
high-GPA subgroup, youth offered treatment and those completing the internship 
were both more likely to attend a two-year college than the control group (by 9 and 18 
percentage points, respectively, at the two-year mark). The program increased the 
probability of persistence only for the low-GPA subgroup. Specifically, for the low-GPA 
subgroup the program increased the probability of attaining a two-year degree or 


being enrolled in their third year by 8 percentage points. 


= Employment, wages, and savings: We estimated the impacts on employment, wages, and savings 
because measuring these outcomes for youth requires a longer time horizon, especially for 
youth still enrolled in some sort of postsecondary institution. We did not find significant effects 
on the probability of having a job, wages, or money accumulated, and this finding persisted at 
the one- and two-year marks. However, we found that participation in the Urban Alliance 
program led to a decrease in the probability of having a job for males at the one-year mark, but 
not at the two-year mark. This finding likely indicates that Urban Alliance encouraged males to 


attend college and therefore away from working after graduating high school. 


What Are the Study's Implications for Policy and 
Practice? 


= Adual-purpose program: The Urban Alliance provides services that would be considered key 
components of an effective employment intervention (US Department of Labor et al. 2014); in 
particular, Urban Alliance provides paid employment, soft skills training, mentors, and 
postprogram support. Nonetheless, the Urban Alliance program serves to give youth an 


appreciation for what they can attain professionally if they go to college. 


A sizable share (31 percent) of program youth did not go to college. For youth focusing on 
employment, the program may need to consider additional supports for youth not going to 
college by helping facilitate job attainment after the internship is completed or offering more 


alumni services. 


=" Gender differences: In general, females were more likely to graduate high school than males and 


more likely to attend college. We found that the program had no impact on college attendance 
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or persistence for females, but it had strong impacts for males. On each of these measures of 
college attendance or persistence, males in the program showed outcomes similar to females, 
indicating the program helps close the educational gap between females and males. For 
example, approximately 70 percent of males in the Urban Alliance program attended college, 
similar to females in either the program or control groups, but only 55 percent of control group 


males attended college. 


Targeting: The Urban Alliance program is aimed at, but not restricted to, middle-of-the-road 
students, that is, students with neither high nor low GPAs. As discussed above, we found that 
Urban Alliance shifted the middle-of-the-road students from attending two-year colleges to 
attending four-year colleges. Although we did not find effects for the low-GPA group on college 
attendance, the program increased the probability that youth in this group would persist in 
college. This finding suggests that the benefits of this program for this group may take longer to 


appear. 


Importance of alumni services: Youth from low-income families, particularly first-generation 


college attendees, frequently need support to help advance through college to obtain a degree. 


» — [tis worth acknowledging the sizable drop-off in college attendance. Although 64 
percent of youth in the control group attended college, just 22 percent had either 
completed a two-year degree or enrolled in a third year by the time of this study. The 
treatment group was consistently five to seven percentage points higher, but it still 
showed the same trend, with 69 percent of the students attending college and 28 
percent attaining a two-year degree or enrolling in their third year by the time of data 
collection. 

» We found little connection of program alumni to Urban Alliance alumni services, 
though good measures did not exist at the time we collected data (Urban Alliance now 
tracks alumni involvement to a greater degree). In addition, we found that the Urban 
Alliance group, or certain subgroups, showed greater hard skill and soft skill comfort at 
the one-year mark than the control group, but both hard and soft skill differences 
dissipated over time. As has been seen in other evaluations, youth programs sometimes 
give the participants a head start on certain dimensions, but the control group youth 
catch up. In itself this is not a negative finding, but it indicates the need to provide 


ongoing support to continue growth of the skills youth acquired during the program. 
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= Urban Alliance should be commended for opening themselves up to the rigorous evaluation we 
conducted. Results highlight several areas of promise as well as matters for consideration in 
refining the programmatic model. Having now expanded from the initial two sites evaluated in 
this study to Chicago, Northern Virginia, and a planned fifth site, the program is indeed 
undergoing just such refinement. To support their geographic and programmatic expansion and 
deepening, Urban Alliance recently received a grant from the US Department of Education’s 
Investing in Innovation Fund to validate the findings of this evaluation and extend them to 
Chicago and Northern Virginia. The Urban Institute will again serve as the evaluator, and 
design has begun. In addition to replication, we hope to gather information allowing us to delve 


deeper into what might account for cohort and site differences. 
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Introduction 


Effective programs to help disadvantaged youth become self-sufficient, attend college, and embark ona 
path toward career success are critical. Urban Alliance, headquartered in Washington, DC, serves such 
youth through its High School Internship Program, which provides high school seniors at risk of 
disconnecting from work and school with training, mentoring, and work experience, with the goal of 
helping them successfully transition to higher education or employment after graduation. Youth 
growing up in low-income and low-opportunity communities, such as those targeted by Urban Alliance, 
face formidable challenges in transitioning to adulthood despite some recent efforts to revitalize 
disinvested neighborhoods and reform struggling school systems. Schools in high-poverty areas often 
lack sufficient resources and offer inadequate instruction; moreover, because of family, neighborhood, 
and peer environment factors, low-income children attending these schools have difficulty taking 
advantage of the educational opportunities that do exist (Jacob and Ludwig 2009). By attending college 
and acquiring job skills, youth in these communities increase their chances of future economic stability, 


but they are less likely to do so than their more advantaged peers. 


In Washington, DC, where the cost of living is high, over a quarter (27 percent) of children under 
age 18 live below the federal poverty level. In Baltimore, MD, the share of children in poverty is even 
higher, at 34 percent.? Many of the schools in these cities have poor academic outcomes. In both 
Baltimore and DC public schools, only around two-thirds (69 and 58 percent, respectively) of students 
who enter ninth grade graduate within four years.? The students who make it to graduation are often 
unprepared for life after high school. Many high school seniors in both cities’ public school systems are 
not proficient in core subjects such as math and English. Unsurprisingly, many of DC’s and Baltimore’s 
young residents do not attend college, have limited options for future skill development, and face 


unemployment. 


Since its founding in 1996, Urban Alliance has placed over 2,700 youth in internships, growing to 
serve over 500 interns annually in four sites: Baltimore (since 2008), Chicago (2012), Northern Virginia 
(2013), and its original site, Washington, DC. As part of this expansion process, Urban Alliance 
commissioned the Urban Institute to conduct a six-year, randomized controlled trial (RCT) impact and 


process evaluation of its High School Internship Program. 


This report describes the early adulthood impacts of the Urban Alliance program, including college 


attendance, persistence, and job preparation. For a detailed description of the implementation of the 
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Urban Alliance internship program, see Theodos et al. (2014), and for a discussion of interim impacts, 


see Theodos, Pergamit, Hanson, et al. (2016). 
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Background 


Barriers to Education 


Despite rising overall rates of college attendance in recent decades, many youth from disadvantaged 
backgrounds still do not enroll in or complete any postsecondary education. Compared with white, non- 
Hispanic youth ages 18 to 21, of whom 53 percent were enrolled in college, only 40 percent of African 
American youth and 43 percent of Hispanic youth were enrolled in college in 2014.4 Eighty percent of 
recent high school completers from high-income families enroll in college, compared with 49 percent of 


completers from low-income families.° 


These trends also apply to college completion. Of first-time students starting at four-year 
institutions in 2007, 63 percent of white students graduated within six years, compared with only 41 
percent of African American students and 53 percent of Hispanic students.° The discrepancies are even 
more striking by income. Of financially dependent youth with at least some postsecondary education, 
96 percent of those in the highest family income quartile earned a bachelor’s degree by age 24, but only 


22 percent of those in the lowest quartile did (Pell Institute 2015). 


The lower rates of college entrance and completion for disadvantaged youth have staggering 
consequences for their future careers, lifetime earnings, and economic stability. Median earnings of 
adults age 25 and up with a high school diploma were $27,809; those with a bachelor’s degree earned 
$50,450 (in 2014 dollars).” Over a lifetime, a person with a bachelor’s degree will earn about two-thirds 


more over her working life than a high school graduate.® 


Many factors lead to reduced college access and success for disadvantaged youth. These youth, 
some of whom would be the first in their families to attend college, often lack the support and guidance 
in their homes and communities that are necessary to prepare for and apply to college (Hair et al. 2009). 
One study found the largest predictor of college success to be the intensity and quality of high school 
curricula (Adelman 1999), which are often lacking in high schools in disadvantaged neighborhoods. 
These factors are even higher predictors for students of color than for white students. A follow-up 
study also found high school academic intensity to be the most important predictor (Adelman 2006). 
Other work has found that students enrolled in remedial education courses in college are less likely to 


earn a degree (Wirt et al. 2004) and that taking a rigorous course increases the number of college 
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credits earned and college grade point average (GPA) for students enrolled in four-year colleges (Long, 


Conger, and latarola 2012). 


Other barriers to college are financial: the rising cost of attendance, limited federal financial aid, 
and insufficient financial resources. Average tuition and fees at private ($31,231) or public ($9,139) 
four-year colleges in 2014-15 were more than triple the cost 30 years prior in real dollars (College 
Board 2014). Though there are more federal financial aid programs now than in the past, and they are 
often larger and serve more types of students (Dynarski and Scott-Clayton 2013), aid has not kept pace 
with rising costs, and there have been recent cuts, such as reductions in Pell grants in 2011. Moreover, 
youth—particularly African American, Latino, and low-income youth—often lack awareness of college 
costs and financial aid options (George-Jackson 2015). Among students who leave college after one 


year or less, 31 percent cite financial reasons (Ross et al. 2012). 


Other research suggests that the level of expectation for college attendance in a community 
influences rates of college entrance (Derden and Miller 2014), that contacts with high school 
counselors regarding information on college are associated with applying to college (Bryan et al. 2011), 
and that more frequent parent-youth discussions about education-related issues are associated with 
greater odds of enrolling in a four-year college, though the benefit of these discussions is smaller for 
African American youth than for other youth (Perna and Titus 2005). Youth who rely heavily on peers, 
rather than parents or school personnel, for information on the college transition are less likely to apply 


to selective colleges (Hill, Bregman, and Andrade 2015). 


Youth who do not enter college still need support in preparing for a career. Youth ages 16 to 19 
suffer high unemployment (20 percent in 2014), especially African American (33 percent) and Hispanic 
(23 percent) youth.? Whether from a lack of work experience or job skills training, youth are often not 
ready for entry-level jobs. In one study, 49 percent of youth who graduated high school but did not 
enter college felt that high school left them unprepared for the work habits they would need in the 
workforce, and employers estimated that 39 percent of recent high school graduates were unprepared 
for their jobs (Achieve, Inc. 2005). Though career success is heavily dependent on educational 
attainment, lack of career preparation may also be a barrier to career success for disadvantaged youth, 


especially those who will not complete a four-year college degree. 
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Programs to Promote Success 


Many programs exist to help prepare youth for college and careers. Programs focus on youth with many 
different education and life experiences, including high school students, dropouts, and youth who have 
experience in the juvenile justice or child welfare systems. Some programs coordinate within high 
schools and offer workplace skills and experience as a part of secondary education, sometimes through 
internships inside or outside of the school. Examples are career academies—partially self-contained 
occupationally themed subschools within high schools—and magnet schools. Such programs exist in 
both Wisconsin and Georgia: their Youth Apprenticeship programs operate within schools and combine 
workplace training in the classroom with work experience and mentorship at a jobsite.*° Other 
programs, like Urban Alliance’s High School Internship Program, are run by private organizations or 
social service departments, rather than through the education system. These programs may offer 
internships, job skills training, or both, and participation may even garner credits toward high school 
graduation. Some programs focus on college readiness, aiming to help students graduate high school 
and enroll in college prepared for the challenge; they may offer instruction, tutoring, and academic 
counseling. Another approach is to offer general case management and/or mentoring, with links to 
other supportive education and employment services as needed. Finally, some programs take a holistic 
approach and offer a combination of job training or internship and academic or college preparation 
counseling, perhaps with other types of supports or general mentoring as well. Urban Alliance’s 
program is one example; another is Project Rise, which offers disconnected youth in New York City; 
Newark, NJ; and Kansas City, MO, case management, classroom education, and a paid internship 


(Manno, Yang, and Bangser 2015). 


Evidence from Youth Programs 


A sizable literature exists describing the impacts of these different types of programs designed to 
improve the educational and career outcomes of youth. Although no studies have rigorously evaluated 
a program with Urban Alliance’s unique combination of work experience, training, mentorship and 
coaching, and alumni support, studies have assessed programs offering different combinations of these 
supports. Several studies have not yielded evidence of positive long-term outcomes. However, many 
studies have only tracked outcomes in the short term, and the major federal evaluations of youth 
employment programs have focused on programs geared toward disconnected youth rather than youth 


still in traditional high school settings. Much can be learned from programs such as Urban Alliance, 
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which provides a comprehensive and intensive array of services to students who are still in high school 


and are in danger of becoming disconnected from education or employment. 


Work Experience 


A review of research on the effect of work experience on youth academic and career outcomes, outside 
of any structured program, shows there may be a positive relationship between employment during 
high school and later outcomes (Treskon 2016). Some nonexperimental longitudinal studies have shown 
that holding a job during high school is associated with higher academic success. For example, Light 
(1999) found that students with jobs during high school who worked a moderate number of hours per 
week (less than 20) performed better in school than students who did not work at all. Ruhm (1995) 
found that students working 20 hours per week had significantly higher earnings six to nine years later 
than their peers who did not work during high school. Rothstein (2001) found positive relationships 
between teenage employment and future employment and education: teens with a moderate level of 
work at ages 16 and 17 worked about six more weeks per year at ages 18 to 30 than those who had not 
worked as teens. Furthermore, teens who worked up to 20 hours per week were more likely to have at 
least some college education by age 30. However, later studies have found no positive correlation 


between teen and later adult employment (Rothstein 2007; Tyler 2003). 


Work-Based Learning 


Secondary education programs that connect students to internships, combine learning with a job, or in 
some other way provide youth with an on-the-job learning experience can prove beneficial. In an RCT of 
career academies, Kemple (2008) found that participants experienced higher levels of interpersonal 
support from peers and teachers, and those students who entered school at high risk of dropping out 
were more likely to stay through 12th grade. Eight years after entering the program, participants had 
earnings and employment rates higher than nonacademy students in their high schools. A quasi- 
experimental study using school administrative data and surveys found that students in career 
academies were 9 percent more likely to graduate and 12 percent more likely to attend a 
postsecondary institution than students in general and vocational tracks (Maxwell and Rubin 1997). 
Similarly, studies of career magnet schools, which specialize in one particular career theme (such as 
agricultural science or business), have found that they result in lower dropout rates and increased 


student investment in school (Katz et al. 1995). Findings have been mixed on whether they improve 
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academic achievement (Ballou, Goldring, and Liu 2006; Cobb, Bifulco, and Bell 2009). In comparing the 
quality of school-based employment with outside employment, one nonexperimental study found that 
students report school-based jobs are lower in quality, but these jobs may offer important work 


experiences to youth who would have difficulty finding work on their own (Hamilton and Sumner 2012). 


Some work-based learning programs operating outside schools have been shown to increase the 
academic performance and classroom attendance of participating students while decreasing delinquent 
behaviors outside class. One study found a positive effect on test scores for youth who participated ina 
local government internship compared to a control group (Hamilton and Zeldin 1987). An RCT 
evaluation of New York City’s Center for Economic Opportunity youth literacy program found that 
students with a paid summer internship to complement their literacy, math, and job skills education 
attended more class hours and improved their math grade a full letter grade more than those without 
the internship (NYC Center for Economic Opportunity 2011). A summer youth employment program in 
Boston was found to reduce adverse social behaviors (such as violence and drug use) among 
participants compared to those in a comparison group consisting of the program’s waiting list (Sum, 
Trubskyy, and McHugh 2013). A Chicago program offering high school students paid summer jobs and a 
job mentor found that youth randomly assigned to be placed in a job had nearly four fewer violent- 
crime arrests per 100 youth than youth assigned to a control group who were not placed in a job (Heller 
2014). Arandom assignment evaluation of Youth Corps, a federally funded program providing paid jobs 
for youth ages 18 to 24, with academic support for those needing General Educational Development 
(GED) certification, found no impacts on educational attainment or employment in an 18-month 
follow-up survey. However, participants were 7 percentage points more likely to report that they 


planned to complete at least some college (Price et al. 2011). 


Few studies have evaluated programs combining an internship with other academic or social 
supports. An RCT evaluation of After School Matters, which offers high school students paid 
“apprenticeship-type” experiences in many settings, found no impacts on marketable job skills or 
academic outcomes, but it did find a reduction in problem behaviors and more markers of positive youth 


development among the treatment group (Hirsch et al. 2011). 


The Summer Career Exploration Program in Philadelphia, PA, which provides high school students 
with a summer job in the private sector, preemployment training, and a college student mentor, was 
found in an RCT to have no effect on students’ high school graduation, college enrollment, attitudes 
toward work or school, or sense of self-efficacy. The program’s only measured positive impact was that 
participants were more likely than control group members to enroll in a college preparatory or 


specialized academic program (12 and 8 percent, respectively; see McClanahan, Sipe, and Smith 2004). 
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It is unclear whether programs like the Summer Career Exploration Program (which lasted only for 
the summer) would be more effective if they were longer term. A quasi-experimental study of a Boston 
school-to-career initiative offering youth intensive academic instruction, worksite learning experiences, 
and post-high school supports found positive impacts for youth who participated in the program 
compared with a control group of youth who would have met the program’s eligibility standards had 
they applied. The study found that the program group members were 6 percentage points more likely to 
attend college, with an even more pronounced positive effect for African Americans (Jobs for the 
Future 1998). An RCT of Year Up, a yearlong program for young adults ages 18 to 24 that combines job 
training, a paid internship, mentoring and counseling, and job search or college application assistance, 
found a positive impact on earnings three years after program completion; on average, Year Up 
participants made $2.51 more in hourly wages than members of the control group and about $13,000 
more than control group members in the three years following the program’s completion. However, 
participants were less likely than youth in the control group to be attending college three years after 
completion, though among those in college, participants were more likely to attend full-time and receive 


financial aid (Roder and Elliot 2014). 


Job Training 


Programs that offer youth job training without direct job experience have documented some success. 
Participation in Job Corps, a federally funded program providing vocational training, academic support, 
counseling, and often residential living, was found in an RCT to have short-term impacts on earnings, 
employment, education, and crime. However, after 5 to 10 years these impacts disappeared for the 
sample as a whole, which contained youth ages 16 to 24 at the time of application, with the impact on 
earnings remaining significant only for the subgroup of youth ages 20 to 24 (Schochet, Burghardt, and 
McConnell 2006). Additionally, an impact study of the Job Training Partnership Act, a previous federal 
program, found no positive impacts on earnings or employment 18 months after program entry for 


either male or female out-of-school youth ages 16 to 21 (Bloom et al. 1993). 


Two studies in the early 1990s of a training program for high school dropouts at San Jose’s Center 
for Employment Training found positive impacts on employment and earnings (Burghardt et al. 1992; 
Cave et al. 1993). An RCT evaluation of replications of San Jose’s Center for Employment Training 
found no lasting impact of the program on earnings or employment, but the authors note this finding 
may have been caused by widespread infidelity to the Center for Employment Training program model 


(Miller et al. 2005). 
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Perhaps the most promising recent job training program evaluation is that of the National Guard 
Youth ChalleNGe program. This program, which provides short-term job and life skills training ina 
quasi-military environment and includes follow-up mentoring, demonstrated long-term positive effects 
on employment. After three years, the randomly assigned program group had an employment rate 7 
percentage points higher and earnings 20 percent higher than the control group, and program 
participants were more likely to obtain college credits or a high school diploma or GED (Millenky et al. 
2011). 


Case Management and Mentoring 


Case management and mentoring programs have documented generally positive results, at least in the 
near and medium term. Impact studies of the Big Brothers Big Sisters mentoring program have reported 
mixed findings. One study found that treatment group members skipped half as many days of school, 
had slightly better GPAs, and had an improved concept of their scholastic competence (Tierney, 
Grossman, and Resch 1995). Another study also found improved academic confidence and 
performance, but only at first; impacts disappeared by 15 months (Herrera et al. 2011). Grossman and 
Rhodes (2002) found that youth enrolled in the program for more than 12 months had significant gains 
at 18 months in self-worth, perceived scholastic competence, relationships with parents, and other 


positive social outcomes. 


A specialized Big Brothers Big Sisters program for children of incarcerated parents found treatment 
group youth had higher self-esteem and felt more connected to school, community, and family at 18 
months, but they did not differ in their academic competence or attitudes (US Department of Justice 
2011). A Philadelphia-area program that provides mentoring for all four years of high school found that 
students offered a mentor had college attendance rates in the first two years after high school 
graduation that were 20 percentage points higher than those of their peers Johnson 1999). The Latin 
American Youth Center’s Promotor Pathway Program, which matches youth with a “promotor” who 
provides intensive case management, mentorship, and advocacy was found to have positive impacts in 
several areas. Compared to a control group, treatment group youth were more likely to be in school, 
less likely to have a child, and less likely to have slept in a homeless shelter 18 months postenrollment; 
positive impacts related to educational attainment or employment were not found (Theodos, Pergamit, 


Derian, et al. 2016). 
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College Access and Readiness 


College access and readiness programs have had mixed results as well. Upward Bound, a federally 
funded program lasting up to four years and offering instruction, tutoring, and counseling, was found to 
have no overall impact on high school graduation or college enrollment. However, it was found to 
improve education outcomes for students with initially low educational expectations. These students 
were more than twice as likely (38 versus 18 percent) to enroll at four-year colleges than similar control 
group members (Myers et al. 2004). The random assignment evaluation of the Quantum Opportunities 
Project, which operated in five sites across the country and offered case management, academic 
support, developmental activities, and community service, found no positive impacts, though this 
finding was attributed to poor implementation of the program model and low participation (Schirm, 


Stuart, and McKie 2006). 


Harvill and colleagues’ (2012) meta-analysis of 14 college-access program experimental or quasi- 
experimental evaluations found an average boost to high school graduation rates of 8 percentage 
points. However, when only the three experimental evaluations were considered, the impact was not 
statistically significant. The analysis also found an average increase in college enrollment of 12 


percentage points, whether all evaluations or only the experimental ones were considered. 


Inall, there is evidence to suggest that programs offering underserved youth jobs, job training, 
career-focused education, mentoring, or college readiness activities—or some combination of these— 
may be effective in helping youth achieve better outcomes. However, we know little about the effects of 
intensive initiatives for students still in high school that provide not only a paid job, but also 
individualized support and continual training (Treskon 2016). From the existing evidence, it is difficult 
to determine if the sort of outcomes achieved by the National Guard Youth ChalleNGe program—that 
is, positive impacts on earnings and employment—could be achieved by a program that takes place 
during the school year and does not include a residential component. A rigorous evaluation of the Urban 


Alliance internship program will help us know more. 
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Urban Alliance High School 
Internship Program Model 


Urban Alliance has developed a program model designed to address the organization’s goals of 
empowering underresourced youth to work and succeed through paid internships, formal training, and 
mentoring. Urban Alliance targets its internship program to a subset of high school seniors reached 


through a months-long recruitment process. 


Target Population and Recruitment 


Urban Alliance targets students at selected schools (further described in the Study Participants section) 
they consider to have a high proportion of youth at risk of not connecting to further education or 
meaningful work. The organization seeks out youth who will be in their senior year of high school during 
the upcoming program year. These youth will need to have enough course credits accumulated to allow 
for an early-release schedule during the internship phase of the program. The Urban Alliance program 
targets high school seniors because program staff believe the program is most effective at reaching 
young people during this transitional year; its lessons and curriculum are designed for youth about to 
enter adulthood. In addition, the program targets youth in their senior year because only by that point 


will they have accumulated enough credits to have a shortened school day schedule. 


The program aims to serve middle-of-the-road high school students who maintain a GPA of 2.0 to 
3.0, but it is not limited to that group. Although Urban Alliance leadership believes students with GPAs 
that are too low will generally have insufficient time, resources, and course credits to participate in the 
program and graduate on time, the program often accepts youth with lower GPAs. The program also 
does not exclude youth with high GPAs, though these students often cannot participate in the program 
because, although they may have sufficient credits for an early-release schedule, they are more likely to 
be taking honors and Advanced Placement courses to fill up their schedules. Youth with high grades 


may also have higher skill levels and more external support, so their need for the program may be lower. 


Urban Alliance begins to recruit students for its High School Internship Program in the spring of 
students’ junior year, and recruitment continues into the fall of their senior year. The recruitment 
process differs between cities. In Washington, DC, the organization’s relationship with schools was 


informal at the time of this study, though it has since become more formalized. Urban Alliance presents 
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its program during assemblies or in classrooms to high school juniors at several public and charter 
schools in the city. In Baltimore, the relationship with the school system is formalized, and youth receive 
course credit for participating in the program. School counselors and teachers identify and refer 
students in their schools who they think will benefit most from the program. Many of these youth do not 
formally apply until they start pre-work training. Urban Alliance has also established a formal 
partnership with the local school districts in the Chicago and Northern Virginia sites, and youth receive 


course credit for participating in the program. 


Program Components 


The Urban Alliance High School Internship Program has four primary components: professional and life 


skills training, paid internships, coaching and mentoring, and alumni services. 


Training: Pre-work and Workshops 


Urban Alliance conducts training workshops from late September or early October of each school year 
through the end of July. This training includes three to six weeks (varying by city) of pre-work training 
before the start of the internship. Program participants are expected to attend training for one to one 
and a half hours every day after school during that period. The primary goal of pre-work training is to 
prepare the youth for their internships. Topics include workplace etiquette and culture, as well as hard 
skills such as faxing and Microsoft Excel basics. Urban Alliance also uses these sessions to familiarize 
youth with post-high school education and employment options, financial literacy, and select life skills. 
During pre-work training, youth receive training on job interviewing, which they then use in interviews 


with mentors at their prospective job sites. 


After the internships start in the late fall, youth are expected to attend workshops most Fridays 
after school. Workshops focus heavily on topics related to post-high school planning, financial self- 
sufficiency, and life skills, though they also continue to review workplace-relevant topics. After the 


school year ends, youth attend half-day workshops every Friday. 


Urban Alliance staff also prepare youth for a final presentation that interns give in July, toward the 
end of the program year, at Urban Alliance’s public speaking challenge event. These PowerPoint 
presentations are designed by the youth to describe their recent internship experiences and career 


goals. A volunteer panel of community stakeholders judges the youth, who can receive a $100 prize for 
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performance. Youth can also receive bonuses earlier in the year for participating at other events or 


participating in program activities while waiting on a delayed job placement. 


Paid Internships 


Urban Alliance program staff pair students who complete pre-work training with paid internships based 
on each student’s skill levels, needs, interests, and the range of internships available. Starting in the late 
fall, Urban Alliance participants go to their internships from 2:00 to 5:00 p.m. after school Monday 
through Thursday. This schedule requires that interns obtain permission for an early-release class 
schedule during their senior year of high school. During the summer following graduation (and 
optionally during winter and spring breaks), Urban Alliance interns work full days Monday through 
Thursday. Urban Alliance partners with professional clothing nonprofits such as Dress for Success to 


give interns access to clothing appropriate for the workplace. 


The settings and responsibilities for internships vary, but most are office settings and include such 
tasks as filing, copying, and answering phones. Other jobs include greeting and directing guests in hotels 
or banks. Some interns also work in educational or day care settings. Interns earn a starting hourly wage 
close to their city’s minimum wage ($8.25 in DC and $7.25 in Baltimore during this study, though both 
these hourly rates are now higher). This wage could rise to $10.00 per hour based on interns’ job 
performance and effort, including workshop attendance and communication with their assigned 
program coordinator. For the most part, interns are officially employed and paid by Urban Alliance 


while working at their internship sites, though select job sites pay interns directly. 


Coaching and Mentorship 


Youth receive job mentoring and general coaching as part of the program. In addition to running the 
training workshops described above, front-line staff (program coordinators) maintain coaching 
relationships with each youth assigned to their workshop group. Each program coordinator has a 
caseload of approximately 30 to 35 interns whom they support throughout the program. Coordinators 
track individual student performance in various areas including workshop and job attendance, 
punctuality, workshop homework assignments, academic progress, post-high school planning, and 
progress toward the presentation at the public speaking challenge. Program coordinators also send out 


a weekly e-mail to youth, and youth must check in with program coordinators at least once during the 
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week. If interns are going to be late to work or miss work, they must contact their program coordinators 


and their employers. 


The program coordinators sit down formally with each intern at least three times per year in a one- 
on-one meeting to discuss post-high school planning. They also provide ad hoc support; speak with 
youth before or after workshop sessions; coordinate groups during workshops in which they discuss 
youths’ program-related experiences; and keep in touch via individual phone calls, e-mails, and texts. 
Some youth face serious challenges such as teen pregnancy, domestic or relationship abuse, problems 
with their home life, or housing instability. Program coordinators support youth emotionally and 


connect them with external resources to meet their needs. 


Each intern is also assigned to a “job mentor” or supervisor, who is an employee at the intern’s 
workplace responsible for ensuring the intern has adequate and appropriate work, teaching the intern 
necessary skills, and, ideally, providing opportunities for enrichment and networking within the 
workplace. Job mentors assess interns’ performance in the workplace. They may suggest possible 
termination of an intern’s position if his or her attendance or performance is poor, but the program 
endeavors to resolve all performance issues except the most severe (e.g., time-sheet fraud). When 
performance concerns arise, Urban Alliance staff first establish a work contract with the youth. Only if 
poor performance persists after several intervention attempts will the organization fire the youth and 


ask the intern to leave the program. 


Alumni Services 


As the Urban Alliance program has grown, the program coordinators have increasingly found 
themselves providing informal support to youth who keep in touch after graduating from the program. 
In 2007, Urban Alliance first began offering informal education and career support services to alumni. 
More recently, Urban Alliance formalized this program component by adding regional alumni services 
directors, and in 2016 it established a national alumni director. Through alumni services, Urban Alliance 
aims to prevent program alumni who are college students from dropping out and to link alumni with 
work. Alumni services also provides an avenue for tracking student outcomes after program 


completion. 


Services for alumni include ad hoc individual coaching meetings with youth, a resource room where 
alumni can access job search and education materials, networking opportunities through a website, 


alumni reunions, and connections to paid internship opportunities. 
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Logic Model 


Initially, Urban Alliance measured its success sporadically and informally (Winkler, Theodos, and Grosz 
2009), but the organization developed a formal logic model that details how it expects program 
activities to lead to specific outputs and, ultimately, outcomes for the youth served (figure 1). The logic 
model describes both the four main activities that youth engage in (left-hand column) and a set of 
outputs and related targets (middle column) associated with them. For example, the first activity is to 
place students in professional, paid internships to expose them to the world of work, and one target is 
that 90 percent of interns complete at least 30 hours of pre-work training. Other outputs relate to the 
development of work skills and initiation of post-high school planning. As the logic model 
demonstrates, the majority of expected short- and long-term outcomes (right-hand column) relate to 
postsecondary education. Program staff articulate they hope most youth will complete college before 
becoming employed; they also assert that the employment-readiness training is valuable for those 


youth who elect to enter the labor force rather than attend college or a technical or training program. 


The logic model highlights the short-term, intermediate, and long-term outcomes and indicators 
associated with the various program activities. In its early years, Urban Alliance developed five goals for 
participants in its High School Internship Program. Specifically, youth would (1) improve their hard and 
soft job skills, (2) gain long-term employment experience, (3) graduate from high school, (4) attend 


college or a training program, and (5) identify long-term employment opportunities. 


A final note about the program design relates to its funding: the internship program is financed 
directly by internship sites and philanthropic foundations. Approximately 75 percent of internship 
placement sites, most typically for-profit businesses but also nonprofit and governmental organizations, 
make a donation to Urban Alliance for each intern they hire; in DC the expected amount is $12,500, and 
in Baltimore it is $10,000. This donation is tax-deductible for the for-profit firms. Urban Alliance uses 
this contribution to pay intern wages and to cover the site-level and national costs of services provided 
to the interns. It raises additional funds to cover the cost of placing students at job sites that cannot 
afford the $12,500 donation. Under the current model, a city must have approximately 70 internship 


slots to be sustainable financially. 
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FIGURE 1 


Urban Alliance High School Internship Program Logic Model 


Activities 


1) Skill Training 

Interns receive training in professional soft 
skills, financial literacy and post-secondary 
planning before internship placement 
through Pre-Work and after internship 
placement during Friday Workshop. 

2) Direct Work Experience 

Interns are placed in internships to practice 
their skills and gain access and exposure to 
professional networks. 

3) Mentorship 

Interns are paired with jobsite mentors who 
supervise and provide feedback to work 
tasks, help establish professional networks, 
and advise on professional development and 
growth. 

4) Case Management 

Program Coordinators guide interns’ 
transition into work, liaise between the 
mentor and intern, oversee post-high school 
planning, and resolve development issues as 
needed. 

5) Alumni Services 

Alumni Services teams continue to provide 
support to both disconnected and connected 
alumni. 


6) Employer Support 

Urban Alliance Program Coordinators support 
employers and jobsite mentors, ensuring 
both a rewarding and impactful internship 
experience. 


Source: Urban Alliance. 


Notes: ASD = alumni services department; FAFSA = Free Application for Federal Student Aid; NSC = National Student Clearinghouse; PC = program coordinators; ROI = return on 


Program Outputs 


Skills Training 

* 90% of interns receive 30 hours of pre-work 
training. 

* Regions maintain 75% workshop attendance rate. 

Internship Experience 

* 75% of interns complete the program. 
75% of interns achieve 500 hours of internship 
experience. 
Interns work on average 80% of hours budgeted 
each month. 
80% of interns receive two or more skill set 
evaluations. 

Mentorship 

* 75% of interns report high mentor engagement in 
quarterly surveys. 

Case Management 

* PCs produce weekly tracking notes on each intern. 

* 95% of alumni have completed Post-High School 
Plan, including FAFSA, personal statement, resume, 
and college applications. 

Alumni Services 

* ASD contacts 70% of NSC alumni call list. 

* ASD provides services for 75% of confirmed 
disconnected alumni. 


Employer Outputs 

* Employers report increased diversity, improved 
culture, and staff management training 
opportunities. 
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Short-Term Program Outcomes 

* 75% of interns show growth on 
professional skills assessment. 
85% of alumni immediately connect 
to a sufficiency pathway (college, 
career training, living-wage 


employment) 


Post-Program and Long-Term 


Outcomes 


80% of alumni are “Connected” one 


year post program 


75% of alumni who enroll in college 
persist into their second year 

50% of disconnected alumni are 
reconnected by alumni service. 
70% of alumni who enter 4-year 
college graduate within 6 years of 


entry. 


70% of alumni who enter 2-year 
colleges graduate within 3 years of 


entry. 


Employer ROI Outcomes 
* 80% jobsite retention rate 


investment. Outputs and outcomes for interns are targets among interns placed at job sites, and those for alumni are targets among interns who complete an internship. 
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Evaluation Design 


This section details the study’s approach to random assignment, data collection, and analysis methods. 
In this final report, we seek to answer three confirmatory research questions: 


=" Doyouth who participate in the Urban Alliance program exhibit stronger hard and soft skills 


than control group of youth not enrolled in the program? 


=" Does the Urban Alliance program lead to increased rates of college enrollment and persistence 


for participants compared to a control group youth? 


=" Do Urban Alliance participants have higher rates of employment and earnings than control 


group youth? 


Additionally, we explore changes in college preparation, high school achievement, employment 


earnings, hours worked, and savings. 


Random Assignment 


For the purpose of this study, we assigned 2011-12 and 2012-13 program applicants in Washington, 
DC, and Baltimore at random to a treatment or control group. We combined two program years to 
provide a sufficient sample size to detect impacts on outcomes of interest. The analyses focus solely on 
the Baltimore and Washington, DC, sites; the organization’s expansion to Chicago and Northern 


Virginia occurred after the evaluation had begun. 


We randomized applicants separately at each site at a ratio of two treatments to one control. This 
ratio was required because Urban Alliance needed a sufficient number of youth to enroll in the 


internship program. We did not include any blocking or stratifying variables when randomizing. 


As part of the application, students gave researchers permission to collect their program, high 
school, and postsecondary school data as well as permission to be contacted to complete a survey. 


Consent to participate in the study was not a requirement to receive Urban Alliance services. 


Urban Alliance invited those assigned to the treatment group to participate in the program, which 
began with mandatory pre-work training before assignment to an internship position. Urban Alliance 


did not invite youth in the control group to participate in the training or internship. 
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Data Collection 


Researchers collected quantitative data from a variety of sources summarized below. 


Application Baseline Data 


Urban Alliance High School Internship Program staff gave all applicants a 12-page application form to 
complete. The application requested detailed contact information, demographics, GPA, school 
attendance records (to be completed by school counselors), goals, career interests, work history, 
household structure, one teacher and one nonteacher recommendation, and parental consent forms. 
The application also provided baseline information for the evaluation. Urban Alliance provided Urban 
Institute with the paper applications for all applicants. We entered a subset of fields relevant to the 
study into an electronic database. Unfortunately, item nonresponse was high for some fields in the 
application, making it impossible to reliably analyze household income, receipt of public benefits, 


education level of household members, and recommenders’ assessments of youths’ hard and soft skills. 


Aggregated High School Baseline Data 


We collected aggregated data about the high schools youth attended. We pulled school-level 
performance data for each school for 2011 from the Maryland and Washington, DC, boards of 
education.*2 To understand the relative performance of schools attended by Urban Alliance youth, we 
accessed information not only on the schools they attended, but also on all schools in Maryland and DC 
according to their average 10th grade reading and math standardized test scores, determining each 


school’s percentile rank among schools in that state or district. 


Additionally, we used data from the National Center for Education Statistics 2010 Common Core 
for school-level information about free and reduced-price lunch eligibility and racial composition.‘* We 
linked the Common Core and performance data with Urban Alliance applicant records to better 


understand youths’ educational environments, opportunities, and challenges. 
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Neighborhood Baseline Data 


The American Community Survey provided characteristics of neighborhoods including unemployment 
rates, poverty rates, and racial and ethnic compositions. We matched study participant addresses to 


census tracts and accompanying indicators from the American Community Survey, 2008-12. 


Youth-Level High School Baseline and Outcome Data 


With the help of Urban Alliance, we accessed student-level data from the DC and Baltimore public 
school systems, the DC Public Charter School Board, and individual charter schools. Data gathered 
included GPA, attendance, and other indicators such as whether students were in a special education 
program and whether they graduated high school. When data on GPA were missing, we used counselor- 


reported or student-reported GPAs from program applications. 


Program Implementation Data 


Urban Alliance tracked service delivery data on case management status and participation in program 
activities, noting youth attendance, progress in completing post-high school planning actions such as 
submitting applications for financial aid, and other important indicators. We relied on these indicators 
to define treatment status, intensity, and inform program activities. Urban Alliance also collected 
financial records of biweekly wages paid to interns. We used these records to calculate total earnings 


and to refine how long youth remained employed at their internships. 


Survey Data 


We collected survey data from the control and treatment groups of both study cohorts about their 
educational, employment, and well-being outcomes. The survey also asked participants about their high 
school experiences, postsecondary education preparation, and family members’ educational 
attainment. A subcontractor, SSRS, fielded the survey at roughly one year and three months (referred to 
as the one-year mark) and roughly two years and three months (referred to as the two-year mark) after 
youths’ expected high school graduation date; the survey at the two-year mark captured outcomes 
during the third year after high school graduation. Youth who completed the survey received a $40 gift 


card for their participation. SSRS implemented the survey online and via telephone. To increase 
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response rates, we augmented these efforts with an in-person interviewer to locate and engage youth 
who had not completed the survey online or by telephone. The survey achieved a 77 percent response 
rate at the one-year mark and a 73 percent response rate at the two-year mark. See appendix A for 
more detailed information on the survey methodology and response rates. Appendix B shows baseline 
characteristics for the analysis sample, providing an assessment of differential attrition across the 


treatment and control groups. The survey instrument is reproduced in appendix G. 


Postsecondary Institution Outcome Data 


The National Student Clearinghouse (NSC) provides information on college enrollment for most 
colleges in the United States, including data on date of enrollment and completion of each semester at 


the individual level for each institution attended. 


The analysis also used National Center for Education Statistics’ Integrated Postsecondary 
Education Data System (IPEDS) data. IPEDS provides data on postsecondary institutions, such as 
location, admissions rate, two- and four-year graduation rates, mean standardized admissions test 
scores, retention rate, and net attendance cost. We used these data to assess the quality of institutions 
that youth attended according to the NSC data. The measures of quality we chose were colleges’ 75th 


percentile for SAT scores, retention rates, and graduation rates. 


Analysis Methods 


This study used an RCT approach that allowed us to estimate the causal impacts that the Urban Alliance 
internship program had on youths’ skills, education, and employment outcomes. The impact of the 
internship program can be measured by differences in outcomes between the treatment and control 
groups. In the following subsections we describe our analytic technique for predicting treatment take- 


up and our approach to intent-to-treat (ITT) and treatment-on-the-treated (TOT) analyses. 


We did not include in our analysis variables with a high number of missing observations, such as 
whether youth had a checking or savings account. Among the variables used, only a small number of 
responses were missing, so we did not impute for missing data. Instead, we omitted the few 
observations with missing data from each analysis. GPA and held a job previously were the only 
exceptions for which we included all observations and included a dummy variable for “missing GPA” or 


for “missing held a job” to prevent losing a large number of observations. 
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Predicting Treatment Take-Up 


Throughout the Urban Alliance program year there were high levels of attrition from the program for 
both cohorts. It is therefore important to first consider which characteristics predict youth participation 
before considering the impact of the internship program. To do so, we considered participation levels at 
three key points in the program: beginning of pre-work, completion of pre-work, and completion of the 
internship. To determine predictive characteristics, we first looked at baseline descriptive 
characteristics of the treatment group for those who reached each benchmark compared with those 


who did not, conditional on meeting the prior benchmark. 


Using regression analysis, we estimated the probability of completing each stage of the program for 
those students assigned to the treatment group, controlling for various individual, school, and 
neighborhood characteristics. A logistic model was used with the following underlying variable 


structure: 
yi = Bo + Baindi + Baneighi + ei 


where ind; is a vector of individual characteristics from the application data and high school data 
including program year, gender, previous job experience, whether the student was a parent, family 
structure, site, language of the parent, special education enrollment, and GPA; neigh; is the percentage 
of people in poverty in youths’ neighborhoods; and y; is the latent propensity that an individual will 
reach the benchmark. We do not observe yi directly; rather, we observe a binary variable for whether 


the individual reached a benchmark: 


fl if y, >0 
** 0 y' <0 


These estimates tell us how each characteristic is related to the probability of attending pre-work, 


completing pre-work, and completing the internship. 


Urban Alliance recruited study participants from 38 high schools in Washington, DC, and Baltimore. 
Students from the same high schools likely had related probabilities of attending the program. Because 
students were not randomly assigned within high schools, the share of treatment students varied across 
high schools. This variation cannot be perfectly controlled for by using school-level characteristics. For 
this reason, we used a random effects model to control for unobserved heterogeneity across different 


high schools in the probability of the outcomes.‘ 
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Intent-to-Treat Analysis 


To estimate the impact of Urban Alliance on youths’ skills, education, and employment outcomes, we 
first used the ITT method, which analyzes outcomes based on initial assignment to either treatment or 
control groups. Not all youth in the treatment group completed the internship program, thus the term 


intent to treat. 


By basing analysis off of the exogenous assignment of youth, we can be certain that any effect 
found through this method is causal. This procedure reveals the effect of offering the program to 
interested students, including those who take up the program but drop out and youth who do not show 


up at all. 


To account for sampling variation, we also used a regression-based approach to control for any 
measured differences between treatment and control groups that may have affected the outcome. 
Because control variables must be unaffected by access to treatment, all control variables used in our 
analysis were measured prior to randomization. The ITT estimate is measured as the average individual 
outcomes for the treatment population minus the average individual outcomes for the control 
population. We control for prerandomization covariates by using a regression framework. Specifically, 


the ITT estimate, 87 would be measured using the following regression equation: 
Y¥, = at BT, + B*X, + & 
where 
Yiis the outcome for each randomly assigned individual i 


T, is an indicator equal to 1 for individuals who were assigned to the treatment group and O for 


individuals assigned to the control group 


B" is the parameter of the ITT effect on the outcome (Yi ), the number of population members 


assigned to the treatment population and control population, respectively 
X; isa vector of prerandomization covariates 
B* is the vector of coefficients on the covariate X n 
€ is the regression error term 


Prerandomization covariates included the following: program year, site, gender, percentage of 


youth’s neighborhood living in poverty, held a job before random assignment (Y/N), and junior year 
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cumulative GPA.*° We used a random effects model in these regressions as well to control for 
unobserved heterogeneity across different high schools in the probability of the outcomes. The results 
are generally robust to different specifications, including controlling for high school-level 
characteristics, clustering the standard errors, estimation technique (e.g., generalized least squares or 


logit regression models), and fixed effects models. 


Impacts may vary across subgroups of youth. In particular, we believe the program may have had 
different effects on students with different levels of high school performance, at the two sites, and for 
males and females. For instance, as we show below, the rate of attending a four-year college for youth 
with GPAs of 3.0 to 4.0 in the control group was 72 percent. This result implies a limited potential to 
induce high-achieving students to go to college. In contrast, the rate of attending a four-year college for 
youth with GPAs of 2.0 to 3.0 in the control group was only 46 percent. Thus the potential of the 
program to affect an outcome like attending a four-year college may depend on GPA. To examine these 
effects, we divided the sample into three groups based on junior year cumulative GPA: less than 2.0, 2.0 
to 3.0, and 3.0 to 4.0. We also explored whether outcomes differed for youth in Washington, DC, and 
Baltimore and for males and females. We estimated separate regressions for each GPA group (low, 


middle, high), each site, and each gender. 


Using a procedure developed by Benjamini and Hochberg (1995), we calculated adjusted 
significance levels for having conducted multiple tests of significance on our main results but not for the 
subgroup results, as these are considered exploratory. The tables in the impact section show unadjusted 
significance levels, but we note in the text where and how the adjusted significance levels differ from 


the unadjusted significance levels. 


We estimated standardized effect sizes for each outcome using Cohen’s D standard effects. We 
calculated these effects by dividing the regression adjusted difference between treatment and control 


by the pooled standard deviation of the outcome. 


Treatment-on-the-Treated Analysis 


A different approach to determining programmatic impacts, TOT, estimates the effects of completing 
the program rather than just the effects of access to treatment. Youth who completed the Urban 
Alliance program (defined as completing pre-work and remaining in the internship until June 1) are 
considered “treated youth,” and those offered access to the Urban Alliance program are “treatment 


youth.” 


PATHWAYS AFTER HIGH SCHOOL aa) 


The TOT method allowed us to estimate effects that may have been drowned out because of the 
high levels of attrition from the program. However, it has a potential downside because youth who 
completed the program may have been systematically different from those who did not complete the 
program. Because of these suspected systematic differences, TOT results are likely only internally valid 
to the group that completed the program. High levels of attrition suggest that persistence in the 
program requires certain levels of motivation, accommodating schedules, and other unobservable 
factors. These factors may also influence youths’ decisions to pursue secondary education or 
employment. There is likely a similar group within the control group that would have been more likely to 
complete the program, and ideally, we would compare the outcomes of youth most likely to complete 
the program in the control group with those who did complete the program. Unfortunately, because of 
the unobservable nature of what influences youth to remain in the program, we cannot conduct this 


comparison. 


To correct for this selection bias, we estimated TOT effects by using an instrumental variables 
approach proposed by Angrist, Imbens, and Rubin (1996). For this estimate, known as the complier 
average causal effect, randomization into the treatment group is used as an instrument for actual 
treatment to remove the bias caused by selection into take-up. This methodology assumes a constant 
causal effect. There is also an exclusionary restriction that the program had no impact on those students 
who were randomly assigned to treatment but who did not complete the program. Of the 59 percent of 
youth who did not complete the program, 38 percent never attended pre-work; 37 percent attended 
less than half of the pre-work trainings; 11 percent attended more than half of the pre-work trainings 
but were not given an internship; and 14 percent were placed in an internship but did not complete the 
internship. For the three-quarters of this group who received at most a low dose of pre-work, it is likely 
that the program had little or no impact. However, for the 25 percent who received either most of the 
pre-work trainings or were placed in the internship, the exclusionary restriction places a strong 


assumption on the program’s effect. 


The complier average causal effect is estimated using two-stage least squares. In the first stage, the 
dependent variable (completing the program) is regressed on the exogenous covariates plus the 
instrument (randomization into treatment). In the second stage, fitted values from the first-stage 
regression are plugged directly into the structural equation in place of the endogenous regressor 
(completing the program). In both stages, we used a linear random effects model. We included the same 


exogenous covariates from the ITT regression. In practice these two equations are estimated jointly. 
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Comparison with Interim Report 


The regressions estimated in this report differ from those in our interim report (Theodos, Pergamit, 
Hanson, et al. 2016) in that we now include a site dummy variable in the regression analysis and we 
restrict the one year survey outcomes to only those who completed the two year survey. Including this 
variable and restricting the sample changed only two primary outcomes. The coefficients on taking the 
ACT and on reported hard skills were statistically significant for the full group in the interim report, but 


are no longer significant. 
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Study Participants 


Overview of Baseline Characteristics 


This section presents the characteristics of the study sample overall and separately for the treatment 
and control groups (table 1). The study sample included 1,062 youth who applied to the Urban Alliance 


program and agreed to participate in the evaluation. 


The vast majority of study youth were African American (89 percent), with small shares of Hispanic 
(5 percent), white (2 percent), and “other” race or ethnicity (4 percent).** Most of the study sample were 


US citizens (95 percent). Eleven percent reported being an English language learner. 


About two-thirds of the study sample were female. Given that the recruitment process was not 
aimed at either female or male students in particular, the sources of this difference are unclear. 
However, we found other programs geared toward teens and young adults also typically serve more 
female than male youth. For example, youth in After School Matters were 59 percent female (Hirsch et 
al. 2011), the Summer Career Exploration Program was 62 percent female (McClanahan, Sipe, and 


Smith 2004), and Upward Bound was 71 percent female (Myers et al. 2004).1” 


Urban Alliance applicants typically come from households with low levels of employment, with 
nearly a quarter of students reporting that no adults in their household were employed. Still, three- 
quarters of applicants reported at least some prior work experience of their own, with an average 
experience of just less than 10 months in all jobs combined. Most typically these positions were summer 
jobs, including jobs accessed through the Summer Youth Employment Program in Washington, DC. 
About 4 in 10 youth reported having a checking or savings account, and a greater portion of youth with 
job experience (42 percent) than of youth with no job experience (30 percent) reported having a bank 


account. 
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TABLE 1 


Characteristics of Urban Alliance Applicants, Overall and by Treatment Group 


Characteristic Full sample Treatment Control 
Demographics 

Female 65% 66% 63% 
US citizen 95% 95%* 97%* 
English language learner 11% 11% 11% 
Race and ethnicity 

African American 89% 88% 91% 
White 2% 2% 2% 
Hispanic 5% 6% 4% 
Other 4% 4% 3% 
Family characteristics 

Has a child 4% 5% 4% 
Employed adult in household 77% 77% 78% 
Living arrangement 

Father only 5% 5% 5% 
Mother only 56% 56% 57% 
Other 12% 11% 13% 
Two parents 27% 28% 25% 
Other characteristics 

Had a previous job 75% 76% 74% 
Has a checking or savings account 38% 35%** 42%** 
Money saved $99 $94 $108 
Observations (n) 1,062 700 362 


Source: Urban Alliance High School Internship Program application forms. 
Note: All items had a response rate of 80 percent or more except bank account (71 percent). 
*significant at 10% **significant at 5% ***significant at 1% 
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TABLE 2 
Characteristics of Urban Alliance Applicants’ Neighborhoods 


Characteristic Baltimore Washington, DC Both sites 
Share nonwhite 

<25% 4% 2% 2% 
25% to <50% 7% 4% 5% 
50% to <75% 12% 9% 9% 
>75% 79% 85% 84% 
Share in poverty 

<10% 9% 12% 11% 
10% to <25% 42% 39% 40% 
25% to <40% 40% 33% 34% 
>40% 11% 16% 15% 
Share unemployed 

<5% 3% 3% 3% 
5% to <10% 13% 21% 20% 
10% to <20% 55% 38% 41% 
>20% 30% 37% 36% 
Observations (n) 200 846 1,046 


Sources: Neighborhood characteristics are five-year averages at the tract level from the American Community Survey, 2008- 
2012, US Census Bureau. Tract determinations were based on youth addresses as reported on Urban Alliance High School 
Internship Program application forms. 

Notes: Estimates include applicants assigned to the treatment groups and the control groups. The table does not include 16 
applicants with incomplete address information. 


Applicants typically resided in economically distressed neighborhoods (table 2). More than three- 
quarters (77 percent) lived in a neighborhood with an unemployment rate of 10 percent or higher, and 
roughly half lived in neighborhoods with poverty rates of 25 percent or higher. Maps of program 
applicants’ home locations in Baltimore and Washington, DC, reveal that almost all reside in census 
tracts comprising over 75 percent people of color (dark green shading in figure 2). Most applicants 
resided in communities with higher-than-average poverty, though a sizable portion of applicants lived in 
moderate-poverty areas (medium-light blue shading in figure 2). Almost no youth hailed from low- 
poverty sections of Baltimore or DC. As one executive staff member said, “Some of them are living in 
moderate-income, mostly African American communities, [and] some of them are living in the toughest 
communities in the District.” Generally, staff members believed that youths’ upbringing in largely 
segregated and low-income neighborhoods limits their opportunities for socioeconomic mobility. As 
one senior member explained, most participants “haven't left their neighborhood,” in the sense that 


they have had very little exposure to opportunities found in middle- and upper-class communities. 
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FIGURE 2 
Characteristics of Urban Alliance Applicants’ Neighborhoods 


Percentage poor in Baltimore Percentage nonwhite in Baltimore 
Legend | Legend 
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Sources: Percentage nonwhite and percentage poor are five-year averages at the tract level from the American Community 
Survey, 2008-12. Tract determinations were based on youth addresses as reported on Urban Alliance High School Internship 
Program application forms. 

Notes: Dots represent the number of program applicants within each census tract (one dot equals one applicant for each map). 
The dots are placed randomly within each tract to display the relative distribution of applicants across a region. 


Overall slightly more than a quarter of Urban Alliance applicants attended a charter school, and in 
Washington, DC, about one-third (34 percent) of applicants attended charter schools. Applicants on 
average exhibited passing but not stellar performance in school, as shown in table 3. The average 


cumulative GPA at the end of junior year was 2.7, according to school records if available, or as reported 
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on the application by a school counselor or by the student. A small but nontrivial share (9 percent) of 
students participated in a special education program. Over one-third (37 percent) of applicants had 
attended more than one high school, a pattern often characteristic of high household instability 
(Theodos, Coulton, and Budde 2014). Applicants demonstrated the intention of attaining a 
postsecondary degree, with 90 percent indicating plans to take the SAT or ACT. 


TABLE 3 


Academic Achievement, Educational Attributes, and School Characteristics of Urban Alliance 
Applicants, Overall and by Treatment Group 


Full Level of 
Characteristic sample Treatment Control — significance 
Academic achievement and educational attributes 
Number of other schools attended in past three 
years 0.5 0.5 0.5 
In special education 9% 8% 10% 
GPA at end of junior year 2.7 2.7 2.6 iil 
Has taken or plans to take ACT or SAT 90% 90% 91% 
Attends magnet school 9% 10% 7% ij 
Attends charter school 28% 28% 28% 
School-level characteristics 
Proficient or advanced in reading 44% 45% 43% 
Proficient or advanced in math 40% 41% 39% = 
African American 90% 89% 90% 
Hispanic 6% 7% 6% 
White 3% 3% 3% 
Other 1% 1% 1% 
Eligible for free or reduced-price lunch 77% 77% 79% * 
Observations (n) 1,062 700 362 


Sources: Urban Alliance High School Internship Program application forms for school; National Center for Education 
Statistics data from 2010 for student body demographic information; Maryland State Department of Education and the 
Washington, DC, Office of the State Superintendent of Education from 2011 for math and reading proficiency. GPAs, 
special education status, and some information on school transfers were provided by DC Public Schools, Baltimore 
Public School Board, DC Public Charter School Board, and individual charter schools in DC. 

Notes: All items had a response rate of 80 percent or more except “has taken or plans to take ACT or SAT” (72 percent). 
Variance was because of nonresponse for some items on the application form. 

*significant at 10% **significant at 5% ***significant at 1% 


Urban Alliance applicants attended a mix of 38 public and charter schools. In DC, about two-thirds 
of youth attended one of eight schools, each accounting for 30 to 61 program applicants; in Baltimore, 
three-quarters attended one of seven schools, each serving at least 9 applicants. Most of the schools 
attended in both sites were low performing and attended mostly by students of color (see table 3). In 
fact, almost all schools that Urban Alliance applicants attended were majority African American, though 


some schools in DC had significant Hispanic student contingents as well. About 93 percent of youth 
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attended schools at which the majority of students were eligible for free or reduced-price school 


Junch.28 


Despite these commonalities, the schools attended differed in some ways. Two schools in DC 
enrolling large numbers of Urban Alliance applicants, Dunbar High School (39 youth) and McKinley 
Technology High School (61 youth), exemplify the diversity in school characteristics. Both have over 95 
percent African American student bodies, but at Dunbar, 100 percent of students were eligible for free 
or reduced-price lunch and barely a quarter of students were proficient on District-wide reading and 
math exams. At McKinley Tech, however, just over half the students were eligible for free or reduced- 
price lunch and nearly 90 percent were proficient in math and reading. Overall, 45 percent of Urban 
Alliance applicants attended a school ranking in the bottom quartile of proficiency in reading and math 


in DC or Maryland, with fewer than 10 percent of students at a school in the top quartile. 


Differences between Treatment and Control Groups 


Data from the Urban Alliance application and school data indicate there were few differences across 
the treatment and control groups, as shown in tables 1 and 3. The overall comparability of these two 
groups indicates that randomization was successful for this study. The variables that were different 
statistically at baseline include the following: US citizenship (95 percent treatment, 97 percent control); 
having a bank account (35 percent treatment, 42 percent control); average GPA at the end of junior 
year (2.7 for treatment, 2.6 for control);!? attending a magnet school (10 percent treatment, 7 percent 
control); the schools’ percentage proficient or advanced in math (41 percent treatment, 39 percent 
control); and the schools’ percentage eligible for free or reduced-price lunch (77 percent treatment, 79 
percent control). As can be seen, although achieving statistical significance, the differences are 


generally small and are not jointly significant. 


Baseline characteristics of youth that completed the follow-up survey, overall and by treatment and 


control group, are shown in appendix B, and characteristics by subgroups are shown in appendix F. 
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Program Take-Up and Services 
Received 


In this section, we briefly describe attrition rates and services receipt for youth in the Urban Alliance 


High School Internship Program. 


The Urban Alliance model allows youth to self-select into the program and expects, by design, 
varying levels of attrition during pre-work, though attrition at this stage depends on the site. There is 
substantial attrition in the Urban Alliance internship program, primarily before and during pre-work 
training. Of those students assigned to the treatment group, 22 percent did not attend any pre-work 
sessions. Although these youth were part of the study—and as a result, their attrition matters in terms 
of measuring programmatic impacts—Urban Alliance does not consider those who applied for the 
program but did not show up to pre-work training, as a program exit (i.e., having been a part of the 
program). Of all treatment group youth, one-quarter (25 percent) began but did not complete pre-work 
training. The remaining 49 percent were placed in a job, and most of those (84 percent) completed the 


program. In all, 41 percent of treatment group youth completed the program (table 4). 


We estimated predictive models that related baseline characteristics of the youth and program to 
the likelihood that youth would complete each of the program stages. Completion did not vary by site, 
gender, parenting, employment history, family structure, language spoken at home, neighborhood 
poverty, or taking special education courses. The probability of completing an internship was 19 to 23 
percentage points higher for those students with GPAs of 2.0 to 4.0 than for those with GPAs below 2.0. 
Additionally, youth from the 2011-12 cohort were 14 percentage points more likely to complete the 
program than youth from the 2012-13 cohort. 


In addition to looking at overall take-up, we also looked at which factors influenced take-up at each 
stage: attending pre-work, completing pre-work, and completing the internship (table 5). Youth who 
previously held a job had a probability of attending pre-work roughly 9 percentage points higher than 
their counterparts. Taking special education courses and being a parent were both negatively 
associated with attending pre-work. Program year 2011-12 was associated with a higher rate of 
showing up to pre-work than program year 2012-13. Conditional on attending pre-work, youth in DC 
were less likely to complete pre-work relative to Baltimore youth. The probability of completing pre- 
work training was 26 to 28 percentage points higher for those students with GPAs of 2.0 to 4.0 than for 


those with GPAs below 2.0. DC youth were more likely to successfully complete their internships. 
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Additionally, a reduction in the caseload of a youth’s program coordinator by 10 youths decreased the 


probability of attrition at the internship stage by 9 percentage points. 


TABLE 4 


Program Attrition, by Site and Cohort 


Washington, 
2011-12 2012-13 DC Baltimore 

Stage in program All (both sites) (both sites) | (bothcohorts) (both cohorts) 
Application accepted (n) 700 310 390 581 119 
Attended pre-work 78% 88% 69% 79% 70% 
Completed pre-work 52% 62% 45% 51% 57% 
Placed at a job 49% 59% 40% 47% 55% 
Completed program 41% 51% 33% 41% 42% 


Source: Urban Alliance program data. 


Note: “Completed program” is defined as remaining in the Urban Alliance program until June 1. 


TABLE 5 
Probability of Take-Up 


Probability of 


Probability of 


attending pre- completing Probability of completing the 
work pre-work program 
Conditionalon Conditional on 
attending pre- completing 
Variable Unconditional work pre-work Unconditional 
Female 0.047 -0.059 0.011 0.002 
(0.033) (0.046) (0.046) (0.040) 
Student is a parent -0.157** -0.147 0.063 -0.122 
(0.069) (0.112) (0.135) (0.099) 
Previously held job 0.086** 0.023 -0.039 0.039 
(0.038) (0.052) (0.058) (0.047) 
Single-parent family -0.025 -0.007 0.003 0.005 
(0.040) (0.051) (0.053) (0.046) 
Other family structure -0.050 0.064 -0.101 0.002 
(0.058) (0.080) (0.071) (0.070) 
Parents speak language other than 
English -0.030 -0.015 0.045 0.032 
(0.058) (0.086) (0.096) (0.075) 
Percentage poverty in 
neighborhood 0.002* 0.001 -0.001 0.001 
(0.001) (0.002) (0.002) (0.002) 
Taking special education courses -0.142*** -0.0106 0.007 -0.098 
(0.055) (0.088) (0.096) (0.076) 
GPA: 3.0 to 4.0 -0.031 0.283*** 0.065 0.189*** 
(0.051) (0.069) (0.082) (0.070) 
GPA: 2.0 to <3.0 0.043 0.257*** 0.071 0.230*** 
(0.047) (0.064) (0.078) (0.065) 
2011-12 cohort 0.186*** 0.046 0.108** 0.139*** 
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Probability of | Probability of 
attending pre- completing Probability of completing the 
work pre-work program 


Conditionalon Conditional on 
attending pre- completing 


Variable Unconditional work pre-work Unconditional 
(0.036) (0.046) (0.055) (0.040) 
DC 0.043 -0.220*** 0.173* -0.058 
(0.054) (0.080) (0.090) (0.071) 
Caseload -0.088** 
(0.044) 
Observations (n) 690 520 360 690 


Sources: Urban Alliance High School Internship Program application forms and Urban Alliance program data. 

Notes: Estimates are marginal effects from a logit regression. Standard errors are given in parentheses. “Completing the program” 
is defined as remaining in the Urban Alliance program until June 1. Reference group for GPA 3.0 to 4.0 and GPA 2.0 to <3.0 was 
GPA <2.0. 

*o<.1**p<.05***p<.01 


Our process evaluation revealed many reasons why youth exited the program. Youth principally 
cited competing priorities, such as athletics or extracurricular activities, class schedules, lack of interest 
in the training, family or personal issues, relocation, and cost of transportation as reasons for attrition. 
During the internship, only a small proportion of youth exited the program. Some youth were fired from 
their internship, primarily for poor attendance; however, some youth exited for positive reasons, such 


as to pursue other educational or job opportunities. 


In addition to estimating attrition, we also estimated whether access to the Urban Alliance program 
led more treatment group youth to receive educational and job services (table 6). It may seem 
definitional that the program led to increased service take-up, but many youth already receive college 
and career preparatory services in their high school or through after-school programs. (We have no 
quality or intensity measures for these other high school or after-school programs.) We found that the 
differences between the treatment and control group of receiving college help and job help were 
statistically significant, but relatively small, at 8 percentage points for college help and 13 percentage 
points for job help. For those students who completed the Urban Alliance program, the differences 
were roughly double, at 17 percentage points and 26 percentage points, respectively. Receiving college 
and job help was prevalent among youth in the control group: 82 percent reported receiving job help, 
and 85 percent reported receiving college help. In comparison, 95 percent of the youth in the treatment 


group reported receiving job help, and 93 percent reported receiving college help. 
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TABLE 6 


Services Received 


Mean ITT 
Difference Regression 
Outcome Treatment Control in means adjusted 
Received job help 0.95 0.82 0.12*** 0.128*** 
(0.031) 
Received college help 0.93 0.85 0.08*** 0.081*** 
(0.027) 


Sources: Urban Alliance program data (control variables) and interim outcome survey (outcome variables). 

Notes: Standard errors are given in parentheses. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 
were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who 
were not accepted into the program. The regression-adjusted models included the following control measures: program year, 
gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

*significant at 10% **significant at 5% ***significant at 1% 
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Program Impacts 


In this section, we present the impact findings of Urban Alliance on skill development, educational 
achievement, and employment outcomes. We describe outcomes approximately one and two years 


after expected high school graduation for each youth. 


For each outcome, we present the means for the treatment and control groups, the regression- 
adjusted ITT estimate, the ITT standardized effect size, and the regression-adjusted TOT estimate. The 
adjusted ITT estimate is an estimate of the size of the difference between the treatment and control 
groups as assigned. We also report the level of significance as a measure of the strength of the evidence 
that the estimate differs from zero. The ITT standardized effect size provides a measure of the 
magnitude of the effect in standardized form, which allows comparisons across different outcomes. 
Finally, the TOT estimate measures the difference between students who completed the program and 
students in the control group. When estimating the TOT, we control for selection into participation by 


using an instrumental variable approach. 


We estimated the impacts of the Urban Alliance program by using regression analysis at the one- 
year mark and the two-year mark. In each regression, we controlled for demographic and educational 
characteristics. Although youth outcomes within a school may vary, youth from the same schools often 
have similar educational outcomes. To account for within-school correlation, we estimated both the ITT 
and TOT effects by using a random effects model at the school level. In general the results are robust to 
alternate specifications including fixed effects or clustered standard errors, although none of the full 
sample effects are robust to a Benjamini-Hockberge multiple comparisons adjustment. We estimated 
the impact of the program for the full group and for three sets of subgroups: by site, by gender, and by 
GPA group. Because of small sample sizes, we did not estimate impacts separately for Baltimore or 
students with GPAs below 2.0 for outcomes based on survey data or IPEDS data. However, we were 
able to estimate impacts for these groups for outcomes based on NSC data. Full-group results at the 
one-year and two-year marks are in the main body of the report. The results of the subgroup analyses 
can be found in appendix C and D with the full-group results repeated. Appendix E provides 


descriptions of the reported outcomes. 
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College Preparation and High School Achievement 


At the one-year mark, we examined the impacts of Urban Alliance on college preparation and high 
school achievement on the full group and on subgroups. We present the results for the full group in 
table 7, and results for the subgroups can be found in appendix C and D. The college readiness outcomes 
[took the SAT, took the ACT, filled out the Free Application for Federal Student Aid (FAFSA), reported 
comfort with the FAFSA and other scholarships, and reported applications to college] were measured in 
the one-year follow-up survey. The high school achievement outcomes (graduation, cumulative GPA, 


absences senior year, and suspensions senior year) were measured through high school transcript data. 


For the full group, we found that Urban Alliance had an impact on youth’s self-reported comfort 
with the FAFSA and other scholarships, though the magnitude of this effect was just under what is 
considered “small.” We did not find any statistically significant impacts on other measures of college 


preparation such as taking the SAT, taking the ACT, or whether the youth applied to college. 


At the subgroup level, we found similar results for youth in DC, for females, and for those with 
GPAs above 3.0. For these subgroups, the only statistically significant impact was for self-reported 
comfort with the FAFSA. For young men, although we did not find a significant effect on any measure of 
college readiness, the magnitude of the effect was considered meaningful for reported comfort with the 
FAFSA and scholarships and reported application to college. Specifically, the estimated increase in 
probability of applying to college was 6 percentage points. This discrepancy between the measures of 
significance and magnitude can occur when sample sizes are too small to achieve significance, but the 
magnitudes are large enough to suggest an effect exists. We did not find any impacts on college 


readiness measures for those with GPAs between 2.0 and 3.0. 


We did not find impacts on any measure of high school achievement for the full group. At the 
subgroup level, consistent with the full-group estimates, we found no impact on high school 
achievement for any subgroup except for males and those with GPAs of 2.0 to 3.0. For males, we found 
that the program increased the probability of graduating from high school by 4 percentage points for 
the ITT estimate and by 8 percentage points for the TOT estimate. For the GPA 2.0 to 3.0 group, we 
found that those who completed the program were more likely to be chronically absent senior year by 


10 percentage points for the ITT estimate and by 20 percentage points for the TOT estimate. 
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TABLE 7 


College Readiness and High School Achievement Impacts at the One-Year Mark 


Mean ITT TOT 
IV 

Outcome (data Observations Regression Standardized regression 
source) (n) Treatment Control adjusted effect size adjusted 
College Preparation 
Took SAT (survey) 555 0.907 0.911 es 0.049 ae 
Took ACT survey) 555 0.443 0.352 a 0.116 coe 
Filled out FAFSA 548 0.931 0.921 -0.002 -0.009 -0.004 
(survey) (0.021) (0.046) 
Comfort with FAFSA 541 3.595 3.466 0.119** 0.199 0.244** 
and scholarships 
(survey) (0.052) (0.108) 
Applied to college 555 0.942 0.919 0.008 0.033 0.005 
(survey) (0.021) (0.043) 
High School 
Achievement 
Graduated high school 939 0.980 0.958 0.012 0.070 0.023 
(HS data) (0.010) (0.025) 
Suspended senior year 955 0.077 0.096 -0.011 -0.038 -0.037 
(HS data) (0.018) (0.045) 
Chronically absent 955 0.322 0.312 0.049 0.106 0.110 
senior year (HS data) (0.032) (0.070) 
Cumulative GPA (HS 926 2.655 2.555 -0.013 -0.021 -0.031 
data) (0.025) (0.060) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, and individual charter schools in DC. 

Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the 
results were estimated using random effect generalized least squares. For dummy outcomes, the results were estimated using a 
random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who were accepted into 
the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted 
into the program. Treatment on the treated (TOT) compared outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures: program year, a 
site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 


Skill Development 


We estimated the impact of Urban Alliance on youths’ skill development at both the one- and two-year 
marks. We present the results for the full group in table 8, and results for the subgroups can be found in 


appendix C and D. The skill development outcomes (self-reported hard skill comfort and self-reported 
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soft skill comfort) were measured using the one-year and two-year follow-up surveys. Hard skill 
comfort is measured as self-reported comfort level performing general office work, such as using Excel, 
making photocopies, etc. Soft skill comfort is measured as self-reported comfort level speaking with 
adult coworkers, writing professional e-mails, making a presentation, dressing professionally, 
completing work assignments on time and getting to work on time (see Appendix E). In the two-year 


survey, we included two additional measures of soft skills: time management and goal setting. 


For the full group, we found a significant impact of the program on reports of soft skills at the one- 
year mark. However, by the two-year mark the size of the effect on soft skills had diminished. The fading 
of the effect appeared to be driven not by a decrease in the reported soft skill comfort of the treatment 
group, but rather a relatively larger increase in the reported soft skill comfort of the control group. 
Although the average reported soft skill comfort for youth in the treatment group grew slightly (from 
3.73 to 3.78), the reported soft skill comfort for youth in the control group grew more (from 3.63 to 
3.70). We did not detect any significant impacts of Urban Alliance at the two-year mark on goal setting 


or time management. 


At the subgroup level, we saw a similar pattern of impacts on at least one measure of skill 
development during the first year that faded by the second year, except for males. For females, there 
were significant impacts on soft skill comfort and hard skill comfort, but both effects faded by the 
second year. For both the GPA 2.0 to 3.0 group and for DC, there were significant impacts on soft skills 
at the one-year mark that faded by the two-year mark. For each of these groups, the fading of the 
impact was largely driven by gains in skill comfort from the control group rather than losses in skill 
comfort from the treatment group. The exception to this pattern of fading impacts was for males. We 
did not detect a statistically significant impact on soft skill comfort at the one-year mark, but found a 
significant effect at the two-year mark, with the effect growing from 0.07 to 0.23. This change was 
driven both by a decline in the average reported soft skills of males in the control group (from 3.69 to 
3.63) and by an increase for males in the treatment group (from 3.74 to 3.85). These results for males 


are Statistically significantly different than those for females. 
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TABLE 8 


Skill Development Impacts 


Mean ITT TOT 
IV 
Outcome (data Observations Regression Standardized regression 
source) (n) Treatment Control adjusted effect size adjusted 
Hard skill comfort 
(survey) 
One year 467 3.656 3.536 0.098 0.144 0.200 
(0.063) (0.127) 
Two years 467 3.656 3.605 0.040 0.063 0.082 
(0.060) (0.121) 
Soft skill comfort 
(survey) 
One year 461 3.731 3.637 0.091** 0.222% 0.182** 
(0.038) (0.077) 
Two years 461 3.781 3.705 0.075* 0.154 0.150* 
(0.046) (0.091) 
Goal setting 506 4.160 4.207 -0.020 -0.018 -0.040 
(survey) (0.101) (0.205) 
Time 515 3.708 3.666 0.072 0.068 0.145 
management 
(survey) (0.095) (0.198) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: IV = instrumental variable. Standard errors are given in parentheses. For continuous outcomes, the results were estimated 
using random effect generalized least squares. For dummy outcomes, the results were estimated using a random effects logit 
model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who were accepted into the program (but who 
may or may not have completed the internship) with a control group of individuals who were not accepted into the program. 
Treatment on the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in 
the control group. The regression-adjusted models included the following control measures: program year, a site dummy, gender, 
neighborhood percentage poverty, previously held a job, and junior year GPA. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 


College Attendance, Quality, and Persistence 


We explored the extent to which Urban Alliance affected participants’ college attendance, the quality of 
the colleges they attended, and their persistence in college. Using NSC data, we measured the share of 
youth who attended and persisted at any college, a four-year college, or a two-year college. By linking 


IPEDS data to NSC attendance data, we were able to assess three proxy measures for college quality: 
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the 75th percentile SAT score for colleges that youth enrolled in and those colleges’ retention and 
graduation rates. The results using the IPEDS data were only available for those who attended college. 
We present results on attendance and college quality at the one- and two-year marks and on 
persistence at the two-year mark in tables 9 and 10. Results at the subgroup level are available in 


appendix C and D. 


For the full-group sample, we did not detect impacts on any of the college attendance or 
persistence measures, including both two- and four-year schools, under either the ITT or TOT 


framework or at the one- or two-year mark (table 9). 


At the subgroup level, the program had large and significant impacts on college attendance and 
persistence for males. Males in the treatment group were 10 percentage points more likely to attend 
college at the one-year mark, and the effect grew to 12 percentage points at the two-year mark. The 
TOT impact was more than two times greater, at 23 percentage points at one year and 28 percentage 
points at two years, for those who completed the program. The magnitude of these increases is 
meaningful considering only about 55 percent of males attended college in the control group. Further, 
this increase in college attendance for males appeared to be driven entirely by an increase in four-year 
college attendance, with estimated increases of 10 percentage points based on the ITT estimate and 
increases of 23 percentage points based on the TOT estimate at the one- and two-year marks. In 
addition, males in the treatment group were more likely to persist in college, with an estimated impact 
of 11 percentage points on the probability of completing two years of college and an estimated impact 
of 10 percentage points on the probability of attaining a two-year degree or persisting into the third 
year. These estimates were more than double for those who completed the program, at 24 and 21 
percent, respectively. These impacts are large considering that only 16 percent of the males in the 
control group completed two years of college, and only 14 percent had either attained a two-year 
degree or persisted into the third year. The results for males on college attendance, completing two 
years of college, completing two years of college at a four-year college, and attending a third year, are all 


significantly different than the results for females. 


The program also had impacts on college attendance and persistence for the GPA subgroups, but 
not for the site subgroups or for females. Although the program had no impact on college attendance 
for those with GPAs below 2.0, it had substantial impacts on persistence, with an estimated impact of 9 
percentage points on the probability of completing two years of college and 8 percentage points on the 
probability of attaining a two-year degree or persisting into the third year. These are large increases 
considering only 5 percent of the control group completed two years of college and less than 4 percent 


attained a two-year degree or persisted into their third year. The GPA 2.0 to 3.0 group, although not 
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more likely to attend college overall, was more likely to have attended a four-year college at the one- 
and two-year marks. At the two-year mark, the probability of attending a four-year college increased by 
9 percentage points for those in the treatment group and by 17 percentage points for those who 
completed the program. These results for the GPA 2.0 to 3.0 group on four-year college attendance are 
significantly different from the results for the GPA 3.0 to 4.0 group, but are not significantly different 
from the GPA 0.0 to 2.0 group. The GPA 2.0 to 3.0 was also less likely to have attended a two-year 
college at the one-year mark, although this effect faded by the two-year mark. It appears the program 


moved this group from attending two-year colleges to attending four-year colleges. 


For the GPA 3.0 to 4.0 group there was an impact on the probability of attending a two-year college 
at both the one- and two-year marks. At the two-year mark, the probability of attending a two-year 
college increased by 9 percentage points for those in the treatment group and by 18 percentage points 
for those who completed the program. This result was significantly different from the result for the GPA 
0.0 to 2.0 and the GPA 2.0 to 3.0 groups. For this group, there was no difference in overall college 
attendance; however, there was a decline in the probability of attending a four-year college, although 
this was not statistically significant. For the GPA 3.0 and above group, Urban Alliance appears to have 
had the opposite effect as for the GPA 2.0 to 3.0 group, moving them from four-year colleges to two- 


year colleges. 


TABLE 9 


Education Impacts 


College Enrollment and Persistence 


Mean ITT TOT 
IV 
Outcome (data Observations Regression Standardized regression 
source) (n) Treatment Control adjusted effect size adjusted 
Attended college 
(NSC) 
One year 1,043 0.637 0.583 0.013 0.026 0.028 
(0.032) (0.073) 
Two years 1,043 0.691 0.642 0.011 0.023 0.027 
(0.030) (0.071) 
Attended four- 
year college (NSC) 
One year 1,043 0.534 0.453 0.024 0.049 0.064 
(0.030) (0.072) 
Two years 1,043 0.579 0.494 0.028 0.057 0.076 
(0.030) (0.071) 
Attended two- 
year college (NSC) 
One year 1,043 0.107 0.133 -0.012 -0.036 -0.026 
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Mean ITT TOT 


IV 
Outcome (data Observations Regression Standardized regression 
source) (n) Treatment Control adjusted effect size adjusted 
(0.018) (0.049) 
Two years 1,043 0.162 0.192 -0.015 -0.040 -0.040 
(0.023) (0.059) 
Completed one 1,043 0.522 0.453 0.015 0.029 0.059 
year of college 
(NSC) (0.031) (0.073) 
Completed one 1,043 0.465 0.406 0.000 0.001 0.007 
year of four-year 
college (NSC) (0.030) (0.071) 
Completed two 1,043 0.302 0.244 0.021 0.047 0.049 
years of college 
(NSC) (0.028) (0.067) 
Completed two 1,043 0.286 0.244 0.002 0.006 0.005 
years of four-year 
college (NSC) (0.027) (0.066) 
Attained two-year 1,043 0.278 0.217 0.020 0.046 0.042 
degree or enrolled 
in third year (NSC) (0.026) (0.064) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: IV = instrumental variable. Standard errors are given in parentheses. The regression adjusted results were estimated using 
arandom effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who were accepted into 
the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted 
into the program. Treatment on the treated (TOT) compared outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures: program year, a 
site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 


For the full-group sample, we did not find any impact on the college quality measures of retention 
rate or graduation rate at the one- or two-year marks (table 10). We found a significant impact on SAT 
score at the one-year mark, but that effect faded by the second year with the control group making 
gains on this measure. These measures of quality were only observable for youth who went to colleges 


that report these measures, which may have introduced selection bias. 


At the subgroup level, we saw a similar pattern of an effect on at least one quality measure at the 
one-year mark that faded by the second year. In DC, there was an effect on both the 75th percentile 
SAT score and the graduation rate at year one. For males, there was an effect at the one-year mark 75th 
percentile SAT score. For the GPA 2.0 to 3.0 group, there was an effect on the graduation rate at one 


year. All of these effects faded by the second year as the control group caught up in terms of college 
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quality measures. Only females saw gains from no effect on any quality measures at year one anda 
significant effect on 75th percentile SAT score at year two. This change was driven by decreases in the 


average 75th percentile SAT score of the control group from year one to year two. 


TABLE 10 
Education Impacts 
College Quality 
Mean ITT TOT 
IV 
Outcome (data Observations Regression Standardized regression 
source) (n) Treatment Control adjusted effect size adjusted 
Retention rate of 
college attended 
(IPEDS) 
One year 580 65.408 63.239 0.613 0.045 1.348 
(1.101) (2.416) 
Two years 619 65.273 62.980 1.698 0.125 2.888 
(1.135) (2.417) 
Graduation rate 
of college 
attended (IPEDS) 
One year 578 35.972 32.543 1.406 0.074 3.053 
(1.550) (3.421) 
Two years 621 34.604 32.657 0.360 0.017 -0.484 
(1.707) (3.733) 
75th percentile 
SAT score of 
college attended 
(IPEDS) 
One year 315 1047.124 1004.25 33.414** 0.241% 49.908 
(16.975) (34.805) 
Two years 303 1044.27 1015.57 15.550 0.101 22.950 
(18.561) (39.065) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: IV = instrumental variable. Standard errors are given in parentheses. The results were estimated using random effect 
generalized least squares. Intent to treat (ITT) compared outcomes of a treatment group of individuals who were accepted into 
the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted 
into the program. Treatment on the treated (TOT) compared outcomes of those in the treatment group who completed the 
internship to those in the control group. The regression-adjusted models included the following control measures: program year, a 
site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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Employment, Wages, and Savings 


Finally, we estimated the impact of the Urban Alliance program on youth employment, wages, and 
savings. We present the results for the full group in table 11 and for subgroups in appendix C and D. The 
employment, wages, and savings outcomes (held a postprogram job, currently employed at the time of 
the survey, postprogram wages, and money accumulated) were measured at the one- and two-year 
marks in the follow-up surveys. For youth, who were not employed their wages are included in the 


analysis as zeros; excluding these observations does not change the qualitative result. 


We did not find that Urban Alliance had significant impacts on job attainment, wages, or 
accumulated savings for the full-group sample at either the one- or two-year marks. Given the high rate 
of college attendance, however, positive labor market outcomes will likely develop over a longer time 


horizon than two years. 


Similarly, for the subgroups we found no significant impact on any of the measures, except for 
males. Males in the treatment and treated groups were less likely to be employed than their 
counterparts in the control group at the one-year mark but not at the two-year mark, with males in the 
treatment group catching up in terms of employment. Given their increased probability of attending 
and persisting in college, this finding at the one-year mark likely indicates that Urban Alliance helped 


male youth enter college, specifically four-year colleges, instead of the workforce. 


TABLE 11 


Employment Impacts 


Mean ITT TOT 
IV 
Outcome (data Observations Regression Standardized _ regression 
source) (n) Treatment Control adjusted effect size adjusted 
Ever helda 
postprogram job 
(survey) 
One year 555 0.419 0.485 -0.058 -0.117 -0.121 
(0.042) (0.088) 
Two years 555 0.814 0.811 0.001 0.002 0.003 
(0.033) (0.070) 
Currently 
employed 
(survey) 
Gneyeat 555 0.313 0.370 -0.053 -0.111 -0.116 
(0.040) (0.083) 
aweveus 555 0.457 0.478 -0.022 -0.044 -0.045 
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Mean ITT TOT 


IV 
Outcome (data Observations Regression Standardized _ regression 
source) (n) Treatment —_Control adjusted effect size adjusted 
(0.043) (0.090) 
Postprogram 
wages (survey) 
Oueyear 553 2.554 2.834 -0.179 -0.042 -0.436 
(0.366) (0.761) 
Teves 553 2.556 2.559 -0.047 -0.011 -0.098 
(0.368) (0.766) 
Money 
accumulated 
(survey) 
One year 548 375.645 337.033 39.073 0.035 74.724 
(97.396) (201.845) 
Two years 548 311.085 259.511 46.475 0.055 100.458 
(72.799) (150.215) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: IV = instrumental variable. Standard errors are given in parentheses. For continuous outcomes, the results were estimated 
using random effect generalized least squares. For dummy outcomes, the results were estimated using a random effects logit 
model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who were accepted into the program (but who 
may or may not have completed the internship) with a control group of individuals who were not accepted into the program. 
Treatment on the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in 
the control group. The regression-adjusted models included the following control measures: program year, a site dummy, gender, 
neighborhood percentage poverty, previously held a job, and junior year GPA. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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Implications for Practice and Policy 


The Urban Alliance High School Internship Program strives to change the trajectories of youth who are 
at risk of becoming disconnected, neither attending college nor finding stable employment. It 
intervenes in their lives at a critical juncture—senior year of high school—and offers them training, an 
internship, and mentoring to help them succeed. The youth it targets live in high-poverty areas where 
the majority of people are of color and where there are low levels of high school graduation and college 
enrollment. They have little exposure to high-skilled employment in their families or neighbors. Many of 
these youth face large financial and social barriers to college, and they are also not adequately prepared 
for the workforce. Within this context, Urban Alliance works to provide youth with the knowledge and 
resources they need to overcome these barriers and enter a path toward future educational attainment 
and financial well-being. In this report, we focus on conclusions and implications based on the 


intervention’s impact throughout the study period and afterward. 


A Dual-Purpose Program 


The Urban Alliance program has at its center an internship, which gives the program the appearance of 
being an employment program. Furthermore, as an employment intervention that might prevent 
disconnection among terminal high school graduates, the program has key components considered 
effective (US Department of Labor et al. 2014); in particular, Urban Alliance provides paid employment, 
soft skills training, mentors, and postprogram support. Although a sizable share (31 percent) of youth in 
the treatment group did not go to college, the Urban Alliance program devotes much of its workshop 
time to helping youth prepare for college. For youth focusing on employment, the program could 
consider additional supports such as helping facilitate job attainment after the internship, perhaps by 
offering more alumni services. In fact, Urban Alliance is augmenting its alumni services program to focus 


on helping youth who become disconnected after completing the program. 


Gender Differences 


In the United States as a whole, females and especially minority females are more likely to graduate high 
school than males and more likely to attend college than males. Moreover, nearly two-thirds (65 


percent) of youth in the Urban Alliance program were female and 89 percent were African American. 
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Our analysis indicated that the Urban Alliance program benefits males more than females in terms of 
soft skill development, college attendance, and college persistence. Females in the Urban Alliance 
program showed no significant differences from their counterparts in the control group at the two-year 
mark. However, males in the Urban Alliance program were more likely to attend college and to attend 
four-year colleges. They were more likely to complete two years of college, particularly at a four-year 
college, and to persist into a third year. On each of these measures, males in the program showed 
outcomes similar to females, indicating the program helps close the educational gap between females 
and males. For example, approximately 70 percent of males in the Urban Alliance program attended 
college, similar to females in either the program or control groups, but only 55 percent of control group 
males attended college. These are strikingly large impacts. They raise valuable questions for Urban 
Alliance, including whether male enrollment and retention in the program can be increased so that even 
more youth can obtain the benefits it confers? Also, can the program be differently structured to better 


assist females? 


Target Population 


The Urban Alliance program is aimed primarily at middle-of-the-road students, that is, students with 
neither high nor low GPAs. However, the program does not enforce eligibility criteria coincident with 
accepting only this targeted population. In our analysis, we segmented the sample into three groups 
based on GPA. Confirming results from the interim report, for the middle GPA subgroup, youth offered 
treatment and those completing the internship were both more likely to attend a four-year college than 


the control group at the two-year mark. 


Low-GPA students in the Urban Alliance program did not enter college in higher rates than their 
control counterparts. However, they benefited from the program in that low-GPA treatment youth 
were more likely than low-GPA control youth to complete two years of college and to persist into a 
third year. Furthermore, although not statistically significant, standardized effect sizes indicated low- 
GPA treatment youth were more likely to complete their first two years at a four-year college. These 


differences are attributable to changes in the probability of persisting in college. 


Oddly, high-GPA students appear to have shifted from attending four-year colleges to attending 
two-year colleges, relative to control youth, a result that is consistent with estimates from the interim 


report. 
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Importance of Alumni Services 


As discussed above, youth who do not go to college and risk becoming disconnected after high school 
graduation need continuing help. Youth who go on to college also need ongoing support. For example, it 
is not necessarily the case, given mounting student loan debt burdens, that attending college without 
attaining a degree advantages students. In that light, it is worth acknowledging the sizable drop-off in 
college attendance. Although 64 percent of youth in the control group attended college, only 22 percent 
had either completed a two-year degree or enrolled in a third year by the time of this study. The 
treatment group was consistently 5 to 7 percentage points higher, but still showed the same trend, with 
69 percent attending college and 28 percent attaining a two-year degree or enrolling in their third year 


by the time of data collection. 


Youth from low-income families, particularly first-generation college attendees, frequently need 
support to help advance through college to obtain a degree. We found little connection of college 
students to Urban Alliance alumni services, though good measures did not exist at the time we 
examined this feature. Existing data recorded case management visits between youth and staff, but 
they did not record casual visits from youth or youth attendance at Urban Alliance events; as of 2016, 


such visits are now recorded by the program. 


We also found that the Urban Alliance group showed greater hard skill and soft skill comfort at the 
one-year mark than the control group, but the differences on hard skills and soft skills both dissipated 
by the two-year mark. As has been seen in other evaluations, youth programs sometimes give the 
participants a head start on certain dimensions, but the control group youth catch up. In itself this is not 
a negative finding, but it indicates the need to provide ongoing support to continue growth of the skills 


youth acquire during the program. 


One possibility to consider is greater linkages to colleges and programs that could provide 
additional supports to Urban Alliance students, helping them—especially the male students of color—to 
remain engaged in school. For example, post-secondary college retention programs offer a combination 
of academic advising, mentorship, and study skills or leadership training; such programs are generally 
institution-specific and are offered at many public and private universities across the country 
(Gardenhire et al. 2016). It is possible that these linkages could be coordinated with Urban Alliance’s 


alumni services program so that they work in a complementary fashion. 
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Looking Ahead 


The Urban Alliance should be commended for opening themselves up to a rigorous evaluation. Results 
highlight several areas of promise and also reveal issues for consideration in refining the programmatic 
model. And, having now expanded from their initial two sites in this study to include Chicago, Northern 
Virginia, and a planned fifth site, the program is indeed undergoing just such refinement. To support 
their geographic and programmatic expansion and deepening, Urban Alliance recently received a grant 
from the Department of Education’s Investing in Innovation Fund to validate the findings of this 
evaluation and extend them to Chicago and Northern Virginia. The Urban Institute will again serve as 
the evaluator, and the evaluation design has begun. In addition to replication, we hope to gather 


information allowing us to delve deeper into what might account for cohort and site differences. 
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Appendix A. Survey Methodology 


The survey sample included the entire control group and a randomly selected subset of treatment 
group participants that matched the size of the control group (table A.1). Students were contacted 


regardless of completion of the interim follow-up survey. 


TABLE A.1 
Survey Sample by Assignment Group and Cohort 


Cohort Control Treatment Total 
2011-12 186 186 372 
2012-13 176 176 352 
Both cohorts 362 362 724 


Source: Urban Alliance High School Internship Program application forms. 


The survey administration was primarily managed by a subcontractor, SSRS. SSRS e-mailed an 
invitation to each member of the survey sample who provided a valid e-mail address explaining the 
purpose of the study and the survey. The e-mail highlighted the offer of a $40 gift card for completing 
the survey and invited youth to complete the survey online. SSRS sent one to two follow-up e-mails 
asking youth to complete the survey. After the initial e-mail messages, SSRS mailed letters of invitation 
to all noncompleters that included information on how youth could complete the survey online or by 
phone. SSRS next mailed a second letter of invitation to noncompleters that included a $2 preincentive 
and the offer of a $40 gift card for completing the survey. SSRS sent up to five additional follow-up e- 
mails to youth who had not yet completed the survey. Following the invitation e-mails, SSRS sent two e- 
mails to noncompleters offering a preincentive of a $10 gift card and then a $30 gift card for completing 


the survey. 


SSRS followed up by telephone with those youth who did not complete the survey via the web. SSRS 
trained call center supervisors and interviewers to administer the survey to ensure accurate data 
collection and maximize response rates. Interviewers received written materials prior to survey 
administration that included an annotated questionnaire, information about the goals of the study, 


pronunciation of key terms, and guidance on overcoming obstacles to accurate answers. 


Before asking whether respondents agreed to the survey, respondents were briefed about the 
confidential and voluntary nature of the survey. If respondents agreed to participate, the survey 


proceeded. If respondents did not agree, the interviewer or online survey screen thanked them for their 
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time and reminded them they could return to the survey if they changed their mind. The survey used 
slightly different language for the treatment and control groups: the treatment group was told the 
survey would be evaluating the Urban Alliance High School Internship Program, and the control group 


was told the survey was aimed at recent DC and Baltimore high school students. 


SSRS used contact information from Urban Alliance program applications, including phone 
numbers, e-mail addresses, parent or guardian contact information, and an emergency contact that was 
supplemented with information collected during the first follow-up survey. Additionally, SSRS 
administered a retention questionnaire to respondents of the interim follow-up about six months prior 
to administering the survey. This questionnaire resulted in 47 percent of respondents either confirming 
or correcting contact information. SSRS supplemented contact information with National Change of 


Address data, Facebook searches, and contact information from the National Student Clearinghouse. 


SSRS called the phone numbers provided by Urban Alliance an average of 16 times if they received 
no answer, a busy tone, or an answering machine before ending the phone call attempts. SSRS 
contacted nonresponsive numbers at multiple times of the day and varied days of the week. SSRS 
offered respondents the option to schedule a call-back. An SSRS team experienced in refusal 
conversions called youth who refused to complete the survey in an attempt to persuade respondents to 
complete data collection. SSRS staff also called youth who began but did not complete the survey to 
encourage them to complete the survey. After SSRS sent the e-mail messages and letters and made the 
calls, youth who did not respond to the survey received text messages on their cellular phones and 


Facebook messages, when possible. 


The final component of survey administration was performed by a consultant interviewer who was 
highly experienced in field research and locating survey participants. The interviewer attempted to 
make in-person contact with nonresponders at their last known addresses in Washington, DC, and 


Baltimore and the addresses of their family or emergency contacts. 


The survey for the 2011-12 cohort was open from November 11, 2014, through February 13, 
2015, and from September 1, 2015, through January 11, 2016, for the 2012-13 cohort. The goal was to 


interview youth after they could have begun a third year of college. 
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The final survey achieved a 73 percent response rate across assignment groups and cohorts (table 
A.2). This response rate was maintained across cohorts. The response rate for the treatment group was 
modestly higher (77 percent) than for the control group (69 percent) as was the case in the first 
outcome survey wave. This pattern held across both waves, with 70 percent of the treatment group and 


62 percent of the control group completing both follow-up surveys. 


TABLE A.2 


Response Levels and Rates by Treatment Group and Cohort 


Cohort Control N (%) Treatment N (%) Total N (%) 
Wave 1 Wave 2 Wave 1 Wave 2 Wave 1 Wave 2 

2011-12 140 (75%) 125 (67%) 155(83%)  145(78%) 295 (79%) 270(73%) 

2012-13 130 (74%) 126 (72%) 136(77%) 132(75%) 266(76%) 258(73%) 


Bothcohorts _270(75%) 251 (69%) 291(80%) _277(77%) 561(77%) 528 (73%) 


Source: Final outcome survey. 
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Appendix B. Differential Attrition 


Differential attrition between the treatment and control groups was minimal (table B.1). There were a few differences between the two groups 
in the final survey that were not present at the time of program application. Treatment group youth who completed the survey were more likely 
to have a father present in the household than control group youth (40 versus 31 percent). Control group youth who completed the survey were 
more likely to be enrolled in special education classes than treatment group youth (9 versus 4 percent). Other differences, however, were 
already present at baseline: treatment group youth had higher GPAs at the end of their junior year, and control group youth were more likely to 
have a checking or savings account. Differential attrition at the one-year mark can be found in the interim report (Theodos, Pergamit, Hanson, et 
al. 2016). 


TABLE B.1 
Survey Respondent Characteristics at Baseline, Overall and by Treatment Group 
Full Sample Survey Respondents Survey Nonrespondents 
eneioettetl All Treatment Control pis All Treatment Control beve All Treatment Control Eve 
of sig. of sig. of sig. 
Youth demographics 
(%) 
Female 65 66 63 68 69 67 — 62 63 55 * 
US citizen 95 95 97 7 97 97 97 94 93 98 = 
mee Janeueee 11 4% 11 13 13 12 9 9 10 
Race and ethnicity (%) 
African American 89 88 91 87 85 89 91 90 95 . 
White 2 2 2 2 1 2 2 2 1 
Hispanic 5 6 4 8 10 5 3 4 2 
Other 4 4 3 4 4 4 4 4 2 
Youth family 
characteristics (%) 
Has a child 4 5 4 4 5 3 ~“ 4 4 4 
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Characteristic 


Full Sample 


Treatment Control 

77 78 

5 5 

56 57 

11 13 

28 25 

76 74 

35 42 
$94 $108 


All 
Employed adult in 7 
household 
Living Arrangement (%) 
Father only 5 
Mother only 56 
Other 12 
Two parents 27 
Other characteristics 
Had a previous job 
(%) 75 
Has a checking or 38 
savings account (%) 
Money saved $99 
Academic achievement and educational attributes 
Number of other 
schools attended in 0.5 
past three years 
In special education 9 
(%) 
GPA at end of junior 27 
year 
Has taken or plans to 90 
take ACT or SAT (%) 
Observations (n) 1,062 


0.5 0.5 
8 10 
2.7 2.6 
90 91 
700 362 


Level 
of sig. 


** 


Source: Urban Alliance High School Internship Program application forms. 
Notes: All items had a response rate of 80 percent or more except bank account (70 percent) and “has taken or plans to take ACT or SAT” (71 percent). For categorical variables 
(living arrangement and race), significance of the chi-square test is shown in the first category row. 


*significant at 10% **significant at 5% 


+chi-squared test significant for categorical variable at the 10% level 
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RK, 


significant at 1% 


All 


79 


$94 


2.7 


91 


528 


Survey Respondents 


Treatment Control 
77 81 
5 5 
48 58 
12 11 
35 26 
74 76 
34 47 
$61 $130 
0.4 0.5 
4 9 
2.7 2.6 
90 91 
276 252 


Level 
of sig. 


2k 


2k 


2 


All 


76 


35 
$103 


0.6 


12 


2.7 


90 


534 


Survey Nonrespondents 


Treatment Control 


77 72 
5 5 
61 54 
11 20 
23 20 
78 69 
37 32 
$115 $58 
0.6 0.6 
11 14 
27 25 
90 91 
424 110 


Level 
of sig. 


2k 


Appendix C. Full Sample and Subgroup Impact 


Tables at One and Two Years 


TABLE C.1 


Urban Alliance Program Impacts, Full Sample 


Mean ITT TOT 
Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Received job help (survey) 

One year 547 0.948 0.823 0.129*** 0.411% 0.260*** 
(0.031) (0.057) 

Received college help (survey) 

One year 549 0.934 0.854 0.080*** 0.263% 0.165*** 
(0.027) (0.054) 

Graduated high school (HS data) 

One year 939 0.980 0.958 0.012 0.070 0.023 
(0.010) (0.025) 

Suspended senior year (HS data) 

One year 955 0.077 0.096 -0.011 -0.038 -0.037 
(0.018) (0.045) 

Chronically absent senior year (HS data) 

One year 955 0.322 0.312 0.049 0.106 0.110 
(0.032) (0.070) 

Cumulative GPA (HS data) 

One year 926 2.655 2.555 -0.013 -0.021 -0.031 
(0.025) (0.060) 

Took SAT (survey) 

One year 555 0.907 0.911 -0.014 -0.049 -0.036 
(0.023) (0.049) 

Took ACT (survey) 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

One year 555 0.443 0.352 0.056 0.116 0.115 
(0.040) (0.083) 

Filled out FAFSA (survey) 

One year 548 0.931 0.921 -0.002 -0.009 -0.004 
(0.021) (0.046) 

Comfort with FAFSA and scholarships (survey) 

One year 541 3.595 3.466 0.119** 0.199 0.244** 
(0.052) (0.108) 

Applied to college (survey) 

One year 555 0.942 0.919 0.008 0.033 0.005 
(0.021) (0.043) 

Hard skill comfort (survey) 

One year 467 3.656 3.536 0.098 0.144 0.200 
(0.063) (0.127) 

Two years 467 3.656 3.605 0.040 0.063 0.082 
(0.060) (0.121) 

Soft skill comfort (survey) 

One year 461 3.731 3.637 0.091** 0.222% 0.182** 
(0.038) (0.077) 

Two years 461 3.781 3.705 0.075* 0.154 0.150* 
(0.046) (0.091) 

Goal setting (survey) 

Two years 506 4.160 4.207 -0.020 -0.018 -0.040 
(0.101) (0.205) 

Time management (survey) 

Two years 515 3.708 3.666 0.072 0.068 0.145 
(0.095) (0.198) 

Attended college (NSC) 

One year 1,043 0.637 0.583 0.013 0.026 0.028 
(0.032) (0.073) 

Two years 1,043 0.691 0.642 0.011 0.023 0.027 
(0.030) (0.071) 


Attended four-year college (NSC) 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

One year 1,043 0.534 0.453 0.024 0.049 0.064 
(0.030) (0.072) 

Two years 1,043 0.579 0.494 0.028 0.057 0.076 
(0.030) (0.071) 

Attended two-year college (NSC) 

One year 1,043 0.107 0.133 -0.012 -0.036 -0.026 
(0.018) (0.049) 

Two years 1,043 0.162 0.192 -0.015 -0.040 -0.040 
(0.023) (0.059) 

Completed one year of college (NSC) 

Two years 1,043 0.522 0.453 0.015 0.029 0.059 
(0.031) (0.073) 

Completed one year of four-year college (NSC) 

Two years 1,043 0.465 0.406 0.000 0.001 0.007 
(0.030) (0.071) 

Completed two years of college (NSC) 

Two years 1,043 0.302 0.244 0.021 0.047 0.049 
(0.028) (0.067) 

Completed two years of four-year college (NSC) 

Two years 1,043 0.286 0.244 0.002 0.006 0.005 
(0.027) (0.066) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 1,043 0.278 0.217 0.020 0.046 0.042 
(0.026) (0.064) 

Retention rate of college attended (IPEDS) 

One year 580 65.408 63.239 0.613 0.045 1.348 
(1.101) (2.416) 

Two years 619 65.273 62.980 1.698 0.125 2.888 
(1.135) (2.417) 

Graduation rate of college attended (IPEDS) 

One year 578 35.972 32.543 1.406 0.074 3.053 
(1.550) (3.421) 

Two years 621 34.604 32.657 0.360 0.017 -0.484 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(1.707) (3.733) 

75th percentile SAT score of college attended (IPEDS) 

One year 315 1,047.124 1,004.250 33.414** 0.241% 49.908 

(16.975) (34.805) 
Two years 303 1,044.269 1,015.567 15.550 0.101 22.950 
(18.561) (39.065) 

Held a postprogram job (survey) 

One year 561 0.419 0.485 -0.066 -0.132 -0.121 
(0.042) (0.088) 

Two years 555 0.814 0.811 0.001 0.002 0.003 
(0.033) (0.070) 

Currently employed (survey) 

One year 561 0.313 0.370 -0.060 -0.126 -0.116 
(0.040) (0.083) 

Two years 555 0.457 0.478 -0.022 -0.044 -0.045 
(0.043) (0.090) 

Postprogram wages (survey) 

One year 553 2.554 2.834 -0.179 -0.042 -0.436 
(0.366) (0.761) 

Tweyeers 553 2.556 2.559 -0.047 -0.011 -0.098 
(0.368) (0.766) 

Money accumulated (survey) 

One year 548 375.645 337.033 39.073 0.035 74.724 
(97.396) (201.845) 

Tinos 548 311.085 259.511 46.475 0.055 100.458 
(72.799) (150.215) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 
Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using random effect generalized 
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least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 
were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on 
the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the 
following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. “—“ indicates either the sample size 
was too small or the regression could not be estimated because of collinearity. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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TABLE C.2 
Urban Alliance Program Impacts, Washington, DC 


Mean ITT TOT 
Observations Regression Standardized’ IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Received job help (survey) 

One year 450 0.948 0.822 0.134*** 0.427% 0.274*** 
(0.033) (0.063) 

Received college help (survey) 

One year 460 0.928 0.870 0.062** 0.208% 0.128** 
(0.028) (0.058) 

Graduated high school (HS data) 

One year 769 0.977 0.956 0.011 0.061 0.010 
(0.013) (0.029) 

Suspended senior year (HS data) 

One year 784 0.079 0.111 -0.020 -0.067 -0.064 
(0.021) (0.052) 

Chronically absent senior year (HS data) 

One year 784 0.350 0.352 0.051 0.107 0.122 
(0.037) (0.082) 

Cumulative GPA (HS data) 

One year 755 2.735 2.600 0.010 0.016 0.024 
(0.026) (0.061) 

Took SAT (survey) 

One year 465 0.913 0.908 -0.010 -0.033 -0.023 
(0.025) (0.053) 

Took ACT (survey) 

One year 465 0.480 0.410 0.052 0.104 0.100 
(0.046) (0.095) 

Filled out FAFSA (survey) 

One year 460 0.925 0.921 -0.014 -0.054 -0.031 
(0.025) (0.052) 

Comfort with FAFSA and scholarships (survey) 

One year 453 3.606 3.507 0.102* 0.174 0.208* 
(0.056) (0.116) 
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Mean ITT TOT 
Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Applied to college (survey) 

One year 465 0.953 0.931 0.002 0.010 0.000 
(0.021) (0.044) 

Hard skill comfort (survey) 

One year 398 3.655 3.573 0.081 0.124 0.124 
(0.067) (0.128) 

Two years 398 3.641 3.640 -0.007 -0.011 -0.028 
(0.064) (0.126) 

Soft skill comfort (survey) 

One year 392 3.735 3.635 0.103** 0.244% 0.197** 
(0.042) (0.081) 

Two years 392 3.773 3.709 0.066 0.131 0.126 
(0.050) (0.096) 

Goal setting (survey) 

Two years 428 4.129 4.131 -0.008 -0.007 -0.015 
(0.114) (0.228) 

Time management (survey) 

Two years 434 3.677 3.601 0.081 0.075 0.158 
(0.106) (0.212) 

Attended college (NSC) 

One year 843 0.667 0.601 0.034 0.071 0.074 
(0.035) (0.079) 

Two years 843 0.726 0.662 0.036 0.079 0.083 
(0.033) (0.076) 

Attended four-year college (NSC) 

One year 843 0.610 0.529 0.046 0.094 0.107 
(0.035) (0.083) 

Two years 843 0.659 0.583 0.043 0.088 0.098 
(0.035) (0.081) 

Attended two-year college (NSC) 

One year 843 0.062 0.076 -0.012 -0.049 -0.027 
(0.016) (0.045) 

Two years 843 0.121 0.119 0.003 0.011 0.012 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(0.023) (0.060) 

Completed one year of college (NSC) 

Two years 843 0.571 0.514 0.010 0.020 0.028 
(0.036) (0.081) 

Completed one year of four-year college (NSC) 

Two years 843 0.534 0.482 0.009 0.018 0.024 
(0.036) (0.082) 

Completed two years of college (NSC) 

Two years 843 0.338 0.291 0.015 0.032 0.035 
(0.033) (0.078) 

Completed two years of four-year college (NSC) 

Two years 843 0.329 0.291 0.006 0.013 0.013 
(0.033) (0.078) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 843 0.310 0.259 0.013 0.028 0.032 
(0.031) (0.076) 

Retention rate of college attended (IPEDS) 

One year 509 66.612 64.386 1.639 0.130 3.543 
(1.128) (2.400) 

Two years 550 65.556 63.753 1.546 0.113 2.585 
(1.218) (2.495) 

Graduation rate of college attended (IPEDS) 

One year 510 37.700 34.324 2.658* 0.148 5.741* 
(1.622) (3.496) 

Two years 551 35.626 35.691 -0.766 -0.038 -3.498 
(1.807) (3.804) 

75th percentile SAT score of college attended (IPEDS) 

One year 282 1,050.032 1,001.543 37.513** 0.264% 53.929 

(18.518) (36.915) 
Two years 276 1,046.812 1,017.035 19.737 0.125 17.640 
(20.313) (40.820) 


Held a postprogram job (survey) 
One year 471 0.413 0.456 -0.043 -0.087 -0.097 
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Mean ITT TOT 


Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(0.046) (0.098) 
Two years 457 0.823 0.797 0.020 0.051 0.043 
(0.037) (0.077) 
Currently employed (survey) 
One year 471 0.311 0.336 -0.028 -0.059 -0.069 
(0.043) (0.092) 
Two years 465 0.457 0.470 -0.013 -0.026 -0.027 
(0.047) (0.099) 
Postprogram wages (survey) 
One year 463 2.496 2.512 -0.034 -0.008 -0.094 
(0.398) (0.837) 
Two years 463 2.665 2.510 0.110 0.025 0.231 
(0.408) (0.856) 
Money accumulated (survey) 
One year 459 383.685 294.270 75.633 0.070 137.504 
(103.007) (214.337) 
Two years 459 335.125 219.976 100.043 0.122 209.411 
(78.866) (164.568) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using random effect generalized 
least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 
were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on 
the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the 
following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. Regression estimates are not shown 
for samples of under 100. “—“ indicates either the sample size was too small or the regression could not be estimated because of collinearity. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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TABLE C.3 


Urban Alliance Program Impacts, Baltimore 


Mean ITT TOT 
Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
Received job help (survey) 
One year 89 0.946 0.827 = - = 
Received college help (survey) 
One year 89 0.973 0.788 = - - 
Graduated high school (HS data) 
One year 170 0.991 0.967 — _ _ 
Suspended senior year (HS data) 
One year 171 0.064 0.033 — _ _ 
Chronically absent senior year (HS data) 
One year 171 0.182 0.148 0.027 0.071 0.077 
(0.046) (0.125) 
Cumulative GPA (HS data) 
One year 171 2.273 2.380 -0.074 -0.116 -0.130 
(0.074) (0.166) 
Took SAT (survey) 
One year 90 0.865 0.925 = = _ 
Took ACT (survey) 
One year 90 0.189 0.113 _ = = 
Filled out FAFSA (survey) 
One year 88 0.973 0.922 — _ _ 
Comfort with FAFSA and scholarships 
(survey) 
One year 88 3.514 3.298 _ - - 
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Mean ITT TOT 
Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Applied to college (survey) 

One year 90 0.865 0.868 = _ _ 

Hard skill comfort (survey) 

One year 69 3.667 3.381 — _ _ 

Two years 69 3.778 3.452 = _ - 

Soft skill comfort (survey) 

One year 69 3.696 3.643 = _ = 

Two years 69 3.852 3.690 = - - 

Goal setting (survey) 

Two years 78 4.429 4.500 = = - 

Time management (survey) 

Two years 81 3.961 3.915 — _ _ 

Attended college (NSC) 

One year 200 0.487 0.524 -0.050 -0.099 -0.118 
(0.072) (0.193) 

Two years 200 0.521 0.573 -0.060 -0.120 -0.147 
(0.072) (0.195) 

Attended four-year college (NSC) 

One year 200 0.160 0.195 -0.046 -0.120 -0.120 
(0.053) (0.154) 

Two years 200 0.193 0.195 -0.014 -0.034 -0.031 
(0.056) (0.158) 

Attended two-year college (NSC) 

One year 200 0.328 0.329 -0.004 -0.008 0.003 
(0.068) (0.179) 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Two years 200 0.361 0.439 -0.081 -0.164 -0.200 
(0.070) (0.189) 

Completed one year of college (NSC) 

Two years 200 0.286 0.244 0.027 0.061 0.070 
(0.063) (0.172) 

Completed one year of four-year college 

(NSC) 

Two years 200 0.126 0.146 -0.027 -0.078 -0.078 
(0.048) (0.137) 

Completed two years of college (NSC) 

Two years 200 0.126 0.085 0.033 0.108 0.085 
(0.047) (0.122) 

Completed two years of four-year college 

(NSC) 

Two years 200 0.076 0.085 -0.016 -0.060 -0.051 
(0.039) (0.111) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 200 0.118 0.073 0.041 0.139 0.106 
(0.046) (0.117) 

Retention rate of college attended (IPEDS) 

One year 71 54.805 57.200 = _ _ 

Two years 69 62.342 58.839 = = _ 


Graduation rate of college attended (IPEDS) 
One year 68 20.374 22.428 = = =- 


Two years 70 24.280 16.309 = = - 


75th percentile SAT score of college attended (IPEDS) 
One year 33 1,012.059 1,018.125 — _ — 


Two years 27 1,013.438 1,004.091 _— - - 
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Mean ITT TOT 

Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
Held a postprogram job (survey) 
One year 90 0.459 0.604 _ - = 
Two years 90 0.757 0.868 = = — 
Currently employed (survey) 
Gnewear 90 0.324 0.509 - - - 
Two years 90 0.459 0.509 - _ _ 
Postprogram wages (survey) 
One year 90 2.948 4.154 — = a 
Twoweate 90 1.818 2.757 - - ~ 
Money accumulated (survey) 
One Vear 89 321.108 513.019 - - - 
Two Year 89 148.000 422.210 — - _ 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 
Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using random effect generalized 


least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 


were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on 


the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the 


following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. Regression estimates are not shown 


a4 


for samples of under 100. 


indicates either the sample size was too small or the regression could not be estimated because of collinearity. 
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*significant at 10% **significant at 5% ***significant at 1% 
“small effect *“medium effect ** “large effect 


TABLE C.4 


Urban Alliance Program Impacts, Females 


Mean ITT TOT 
Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
Received job help (survey) 
One year 369 0.948 0.844 0.115*** 0.380% 0.235*** 
(0.036) (0.068) 
Received college help (survey) 
One year 370 0.948 0.878 0.079*** 0.280% 0.151*** 
(0.032) (0.061) 
Graduated high school (HS data) 
One year 619 0.977 0.969 0.002 0.010 -0.011 
(0.013) (0.029) 
Suspended senior year (HS data) 
One year 627 0.066 0.072 -0.006 -0.023 -0.022 
(0.021) (0.050) 
Chronically absent senior year (HS data) 
One year 627 0.363 0.349 0.044 0.092 0.093 
(0.040) (0.086) 
Cumulative GPA (HS data) 
One year 615 2.691 2.633 -0.023 -0.036 -0.054 
(0.031) (0.072) 
Took SAT (survey) 
One year 373 0.908 0.917 -0.010 -0.036 -0.032 
(0.028) (0.060) 
Took ACT (survey) 
One year 373 0.477 0.387 0.073 0.148 0.142 
(0.050) (0.103) 
Filled out FAFSA (survey) 
One year 371 0.943 0.939 -0.001 -0.003 -0.002 
(0.022) (0.050) 
Comfort with FAFSA and scholarships (survey) 
One year 363 3.599 3.471 0.108* 0.184 0.222* 
(0.063) (0.129) 
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Mean ITT TOT 
Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Applied to college (survey) 

One year 373 0.933 0.934 -0.005 -0.021 -0.011 
(0.024) (0.051) 

Hard skill comfort (survey) 

One year 321 3.694 3.510 0.152** 0.227% 0.306** 
(0.075) (0.152) 

Two years 321 3.653 3.621 0.027 0.042 0.056 
(0.073) (0.149) 

Soft skill comfort (survey) 

One year 317 3.726 3.616 0.110** 0.265% 0.220** 
(0.047) (0.096) 

Two years 317 3.749 3.737 0.029 0.059 0.059 
(0.056) (0.114) 

Goal setting (survey) 

Two years 348 4.182 4.135 0.095 0.083 0.207 
(0.126) (0.263) 

Time management (survey) 

Two years 351 3.706 3.619 0.136 0.125 0.287 
(0.119) (0.250) 

Attended college (NSC) 

One year 677 0.641 0.634 -0.031 -0.064 -0.065 
(0.039) (0.089) 

Two years 677 0.688 0.696 -0.044 -0.096 -0.093 
(0.038) (0.086) 

Attended four-year college (NSC) 

One year 677 0.553 0.511 -0.019 -0.038 -0.038 
(0.038) (0.088) 

Two years 677 0.593 0.546 -0.014 -0.028 -0.029 
(0.038) (0.086) 

Attended two-year college (NSC) 

One year 677 0.092 0.128 -0.016 -0.051 -0.026 
(0.021) (0.055) 

Two years 677 0.142 0.189 -0.024 -0.065 -0.054 
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Mean TOT 
Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(0.027) (0.067) 
Completed one year of college (NSC) 
Two years 677 0.542 0.493 -0.000 -0.000 0.014 
(0.039) (0.088) 
Completed one year of four-year college (NSC) 
Two years 677 0.488 0.454 -0.026 -0.052 -0.053 
(0.038) (0.087) 
Completed two years of college (NSC) 
Two years 677 0.316 0.295 -0.019 -0.041 -0.042 
(0.036) (0.082) 
Completed two years of four-year college (NSC) 
Two years 677 0.296 0.295 -0.045 -0.098 -0.102 
(0.035) (0.082) 
Attained two-year degree or enrolled in third year (NSC) 
Two years 677 0.294 0.264 -0.016 -0.036 -0.040 
(0.034) (0.080) 
Retention rate of college attended (IPEDS) 
One year 396 65.315 63.697 -0.470 -0.035 -0.976 
(1.279) (2.783) 
Two years 414 64.807 62.156 1.871 0.138 3.757 
(1.341) (2.788) 
Graduation rate of college attended (IPEDS) 
One year 394 36.303 33.157 0.388 0.022 0.965 
(1.748) (3.799) 
Two years 415 35.539 33.118 -0.631 -0.032 -1.523 
(1.942) (4.075) 
75th percentile SAT score of college attended 
(IPEDS) 
One year 206 1,039.261 1,010.879 20.902 0.157 31.133 
(19.563) (40.213) 
Two years 191 1,039.508 1,000.846 32.017 0.230% 45.737 
(21.595) (43.235) 
Held a postprogram job (survey) 
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Mean ITT TOT 


Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
One year 376 0.405 0.436 -0.031 -0.063 -0.036 
(0.051) (0.108) 
Two years 373 0.836 0.807 0.027 0.069 0.059 
(0.040) (0.085) 
Currently employed (survey) 
One year 376 0.308 0.326 -0.018 -0.039 -0.032 
(0.048) (0.102) 
Two years 373 0.456 0.492 -0.038 -0.076 -0.079 
(0.052) (0.111) 
Postprogram wages (survey) 
One year 372 2.557 2.557 0.053 0.013 0.088 
(0.431) (0.909) 
Two years 372 2.443 2.520 -0.089 -0.022 -0.164 
(0.430) (0.906) 
Money accumulated (survey) 
One year 369 387.088 288.845 98.721 0.092 199.701 
(113.757) (238.185) 
Two years 369 275.420 216.856 74.789 0.106 156.621 
(73.038) (153.916) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using random effect generalized 
least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 
were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on 
the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the 
following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. Regression estimates are not shown 
for samples of under 100. “—“ indicates either the sample size was too small or the regression could not be estimated because of collinearity. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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TABLE C.5 


Urban Alliance Program Impacts, Males 


Mean ITT TOT 
Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Received job help (survey) 

One year 170 0.947 0.782 0.186*** 0.557*%* 0.369*** 
(0.074) (0.107) 

Received college help (survey) 

One year 179 0.904 0.807 0.090* 0.257% 0.183* 
(0.054) (0.111) 

Graduated high school (HS data) 

One year 320 0.986 0.940 0.041* 0.218% 0.076 
(0.024) (0.048) 

Suspended senior year (HS data) 

One year 328 0.098 0.134 -0.019 -0.058 -0.035 
(0.036) (0.088) 

Chronically absent senior year (HS data) 

One year 328 0.237 0.252 0.037 0.087 0.078 
(0.050) (0.116) 

Cumulative GPA (HS data) 

One year 311 2.581 2.418 0.008 0.012 0.033 
(0.043) (0.096) 

Took SAT (survey) 

One year 174 0.906 0.899 -0.028 -0.095 -0.039 
(0.039) (0.085) 

Took ACT (survey) 

One year 182 0.375 0.281 0.058 0.124 0.143 
(0.070) (0.144) 

Filled out FAFSA (survey) 

One year 177 0.905 0.882 0.008 0.027 0.004 
(0.047) (0.099) 

Comfort with FAFSA and scholarships 

(survey) 
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Mean 


ITT 


TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

One year 178 3.585 3.454 0.147 0.234% 0.298 
(0.099) (0.202) 

Applied to college (survey) 

One year 182 0.958 0.888 0.059 0.224% 0.085 
(0.044) (0.081) 

Hard skill comfort (survey) 

One year 146 3.575 3.597 0.014 0.020 0.026 
(0.118) (0.228) 

Two years 146 3.663 3.567 0.076 0.121 0.149 
(0.106) (0.202) 

Soft skill comfort (survey) 

One year 144 3.743 3.686 0.066 0.164 0.123 
(0.068) (0.128) 

Two years 144 3.850 3.631 0.213*** 0.444% 0.409*** 
(0.079) (0.151) 

Goal setting (survey) 

Two years 158 4.111 4.359 -0.243 -0.240 -0.412 
(0.168) (0.317) 

Time management (survey) 

Two years 164 3.714 3.761 -0.006 -0.006 -0.040 
(0.162) (0.312) 

Attended college (NSC) 

One year 366 0.629 0.496 0.097* 0.198 0.226* 
(0.053) (0.131) 

Two years 366 0.696 0.549 0.116** 0.240% 0.283** 
(0.049) (0.129) 

Attended four-year college (NSC) 

One year 366 0.496 0.353 0.099** 0.203% 0.228* 
(0.049) (0.127) 

Two years 366 0.554 0.406 0.103** 0.208% 0.230* 
(0.049) (0.128) 


Attended two-year college (NSC) 


APPENDIX C 


Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

One year 366 0.138 0.143 -0.005 -0.014 0.011 
(0.036) (0.098) 

Two years 366 0.200 0.195 0.010 0.026 0.045 
(0.042) (0.115) 

Completed one year of college (NSC) 

Two years 366 0.483 0.383 0.053 0.107 0.138 
(0.052) (0.131) 

Completed one year of four-year college 

(NSC) 

Two years 366 0.421 0.323 0.052 0.108 0.122 
(0.049) (0.125) 

Completed two years of college (NSC) 

Two years 366 0.275 0.158 0.107** 0.263% 0.241** 
(0.047) (0.119) 

Completed two years of four-year college 

(NSC) 

Two years 366 0.267 0.158 0.098** 0.242% 0.207* 
(0.046) (0.117) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 366 0.246 0.135 0.098** 0.251% 0.207* 
(0.045) (0.112) 

Retention rate of college attended (IPEDS) 

One year 184 65.598 62.161 2.462 0.169 6.343 
(2.231) (5.013) 

Two years 205 66.177 64.774 0.999 0.073 2.173 
(1.980) (4.740) 

Graduation rate of college attended (IPEDS) 

One year 184 35.290 31.143 3.771 0.178 9.126 
(3.166) (7.442) 

Two years 206 32.805 31.644 0.501 0.022 -0.949 
(3.281) (8.097) 


75th percentile SAT score of college attended (IPEDS) 
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Mean 


ITT 


TOT 


Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
One year 109 1,060.549 987.679 56.366* 0.383% 131.443 
(33.857) (82.146) 
Two years 112 1,051.852 1,045.469 -11.161 -0.062 -27.692 
(36.080) (85.765) 
Held a postprogram job (survey) 
One year 185 0.448 0.584 -0.151** -0.303 -0.301** 
(0.074) (0.155) 
Two years 174 0.771 0.820 -0.037 -0.092 -0.084 
(0.063) (0.125) 
Currently employed (survey) 
One year 185 0.323 0.461 -0.160** -0.329 -0.296** 
(0.073) (0.154) 
Two years 182 0.458 0.449 0.019 0.039 0.056 
(0.076) (0.157) 
Postprogram wages (survey) 
One year 181 2.546 3.397 -0.726 -0.159 -1.776 
(0.712) (1.425) 
Two years 181 2.787 2.639 0.236 0.051 0.391 
(0.729) (1.445) 
Money accumulated (survey) 
One year 179 352.398 436.177 -107.585 -0.089 -157.049 
(186.377) (360.583) 
Two years 179 383.541 347.271 70.314 0.066 125.726 
(162.960) (322.874) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 
Notes: Obs. = observations; IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using 
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random effect generalized least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a 
treatment group of individuals who were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not 
accepted into the program. Treatment on the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The 
regression-adjusted models included the following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year 
GPA. Regression estimates are not shown for samples of under 100. “—“ indicates either the sample size was too small or the regression could not be estimated because of 
collinearity. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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TABLE C.6 
Urban Alliance Program Impacts, GPA 0.0 to 2.0 


Mean ITT TOT 
Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
Received job help (survey) 
One year 75 0.939 0.857 — — - 
Received college help (survey) 
One year 75 0.906 0.791 — — — 
Graduated high school (HS data) 
One year 129 0.887 0.902 0.013 0.043 0.049 
(0.057) (0.223) 
Suspended senior year (HS data) 
One year 138 0.140 0.130 0.037 0.108 0.013 
(0.060) (0.253) 
Chronically absent senior year (HS data) 
One year 138 0.581 0.630 -0.044 -0.090 -0.306 
(0.086) (0.348) 
Cumulative GPA (HS data) 
One year 133 1.731 1.701 0.070 0.193 0.295 
(0.059) (0.249) 
Took SAT (survey) 
One year 77 0.765 0.721 _ _ = 
Took ACT (survey) 
One year 77 0.353 0.233 = _ _ 
Filled out FAFSA (survey) 
One year 74 0.781 0.810 — — _ 


Comfort with FAFSA and scholarships 
(survey) 
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Mean 


ITT 


TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

One year 74 3.545 3.573 — _ _ 

Applied to college (survey) 

One year 77 0.735 0.767 — — _ 

Hard skill comfort (survey) 

One year 65 3.700 3.629 - - _ 

Two years 65 3.600 3.686 — — _ 

Soft skill comfort (survey) 

One year 62 3.743 3.729 = - _ 

Two years 62 3.821 3.824 _ - _ 

Goal setting (survey) 

Two years 72 4.290 4.049 = - _ 

Time management (survey) 

Two years 73 3.672 3.631 = = _ 

Attended college (NSC) 

One year 147 0.315 0.333 -0.014 -0.029 -0.031 
(0.086) (0.303) 

Two years 147 0.391 0.386 0.018 0.037 0.095 
(0.089) (0.322) 

Attended four-year college (NSC) 

One year 147 0.196 0.193 0.054 0.136 0.369 
(0.068) (0.265) 

Two years 147 0.239 0.246 0.037 0.086 0.257 
(0.077) (0.282) 

Attended two-year college (NSC) 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

One year 147 0.120 0.140 -0.057 -0.168 -0.387* 
(0.054) (0.227) 

Two years 147 0.174 0.211 -0.062 -0.156 -0.351 
(0.066) (0.264) 

Completed one year of college (NSC) 

Two years 147 0.163 0.175 0.038 0.101 0.277 
(0.062) (0.223) 

Completed one year of four-year college 

(NSC) 

Two years 147 0.120 0.123 0.054 0.163 0.380* 
(0.055) (0.208) 

Completed two years of college (NSC) 

Two years 147 0.098 0.053 0.094** 0.354% 0.457*** 
(0.047) (0.183) 

Completed two years of four-year college 

(NSC) 

Two years 147 0.087 0.053 0.077* 0.301% 0.389** 
(0.045) (0.177) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 147 0.076 0.035 0.075* 0.329% 0.399** 
(0.042) (0.165) 

Retention rate of college attended (IPEDS) 

One year 47 59.655 54.667 = _ = 

Two years 52 59.563 54.550 = _ _ 


Graduation rate of college attended (IPEDS) 
One year 48 22.920 24.657 = = _ 


Two years 52 19.672 20.825 = _ 


75th percentile SAT score of college attended (IPEDS) 
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Mean ITT TOT 


Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
One year 8 1,032.000 911.667 _ _ - 
Two years 10 1,024.000 944.000 — _ - 
Held a postprogram job (survey) 
One year 77 0.294 0.419 = = _ 
Two years 77 0.735 0.767 — — _ 
Currently employed (survey) 
One year 77 0.206 0.209 = ~ 2 
Two years 77 0.500 0.419 = = = 
Postprogram wages (survey) 
One year 76 1.985 2.034 = = _ 
Two years 76 2.156 1.138 — _ _ 
Money accumulated (survey) 
One year 75 341.742 251.119 = - - 
Two years 75 107.000 32.837 — — _ 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 

Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using random effect generalized 
least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 
were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on 
the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the 
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following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. Regression estimates are not shown 
for samples of under 100. “—“ indicates either the sample size was too small or the regression could not be estimated because of collinearity. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 


APPENDIX C Ea 


TABLE C.7 


Urban Alliance Program Impacts, GPA 2.0 to 3.0 


Mean ITT TOT 
Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Received job help (survey) 

One year 259 0.965 0.803 0.182*** 0.586*% 0.300*** 
(0.049) (0.074) 

Received college help (survey) 

One year 259 0.915 0.866 0.038 0.121 0.069 
(0.040) (0.071) 

Graduated high school (HS data) 

One year 482 0.994 0.975 0.016 0.133 0.035 
(0.011) (0.023) 

Suspended senior year (HS data) 

One year 483 0.083 0.093 -0.008 -0.028 -0.020 
(0.026) (0.059) 

Chronically absent senior year (HS data) 

One year 483 0.327 0.253 0.102** 0.225% 0.192** 
(0.046) (0.090) 

Cumulative GPA (HS data) 

One year 474 2.538 2.541 -0.026 -0.074 -0.064 
(0.030) (0.064) 

Took SAT (survey) 

One year 267 0.916 0.953 -0.023 -0.092 -0.035 
(0.030) (0.054) 

Took ACT (survey) 

One year 267 0.441 0.362 0.049 0.100 0.132 
(0.060) (0.110) 

Filled out FAFSA (survey) 

One year 264 0.930 0.944 -0.006 -0.025 -0.008 
(0.030) (0.056) 

Comfort with FAFSA and scholarships (survey) 

One year 260 3.612 3.508 0.077 0.134 0.139 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(0.073) (0.133) 

Applied to college (survey) 

One year 267 0.951 0.961 -0.008 -0.037 -0.017 
(0.024) (0.047) 

Hard skill comfort (survey) 

One year 229 3.653 3.575 0.046 0.070 0.090 
(0.087) (0.157) 

Two years 229 3.605 3.632 -0.029 -0.045 -0.053 
(0.090) (0.163) 

Soft skill comfort (survey) 

One year 226 3.733 3.642 0.093* 0.220% 0.165* 
(0.056) (0.100) 

Two years 226 3.740 3.692 0.058 0.110 0.104 
(0.071) (0.125) 

Goal setting (survey) 

Two years 242 4.068 4.236 -0.173 -0.153 -0.307 
(0.151) (0.273) 

Time management (survey) 

Two years 247 3.638 3.643 -0.006 -0.006 -0.011 
(0.138) (0.249) 

Attended college (NSC) 

One year 507 0.644 0.603 0.027 0.055 0.049 
(0.048) (0.096) 

Two years 507 0.703 0.667 0.023 0.050 0.044 
(0.045) (0.092) 

Attended four-year college (NSC) 

One year 507 0.565 0.448 0.089** 0.179 0.182* 
(0.046) (0.095) 

Two years 507 0.609 0.494 0.085* 0.171 0.169* 
(0.045) (0.094) 


Attended two-year college (NSC) 
One year 507 0.082 0.161 -0.060** -0.183 -0.135** 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(0.026) (0.060) 

Two years 507 0.144 0.213 -0.047 -0.124 -0.104 
(0.032) (0.074) 

Completed one year of college (NSC) 

Two years 507 0.518 0.454 0.029 0.058 0.052 
(0.049) (0.096) 

Completed one year of four-year college (NSC) 

Two years 507 0.479 0.397 0.048 0.098 0.094 
(0.047) (0.094) 

Completed two years of college (NSC) 

Two years 507 0.268 0.236 0.007 0.016 0.011 
(0.038) (0.085) 

Completed two years of four-year college (NSC) 

Two years 507 0.262 0.236 0.002 0.004 0.000 
(0.038) (0.085) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 507 0.244 0.218 0.003 0.006 0.002 
(0.035) (0.083) 

Retention rate of college attended (IPEDS) 

One year 284 62.221 61.108 0.193 0.016 0.349 
(1.502) (3.035) 

Two years 312 63.119 60.816 1.888 0.151 3.090 
(1.524) (3.014) 

Graduation rate of college attended (IPEDS) 

One year 283 32.454 28.010 3.230* 0.218% 6.488* 
(1.810) (3.624) 

Two years 313 32.607 29.307 0.913 0.051 1.300 
(2.147) (4.311) 

75th percentile SAT score of college attended (IPEDS) 

One year 142 999.207 956.278 26.868 0.280% 49.292 

(20.101) (37.250) 
Two years 136 1,002.180 976.159 17.624 0.143 -15.028 


86 APPENDIX C 


Mean 


ITT 


TOT 


Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(24.356) (43.591) 
Held a postprogram job (survey) 
One year 270 0.455 0.496 -0.041 -0.081 0.003 
(0.061) (0.113) 
Two years 260 0.818 0.811 0.026 0.067 0.044 
(0.048) (0.087) 
Currently employed (survey) 
One year 270 0.329 0.394 -0.063 -0.131 -0.044 
(0.060) (0.108) 
Two years 267 0.448 0.449 -0.005 -0.011 -0.011 
(0.062) (0.114) 
Postprogram wages (survey) 
One year 267 2.924 2.942 0.266 0.059 0.525 
(0.560) (1.035) 
Two years 267 2.424 2.794 -0.225 -0.051 -0.250 
(0.553) (1.014) 
Money accumulated (survey) 
One year 267 371.383 243.111 166.896 0.175 301.813 
(119.887) (220.348) 
Two years 267 313.057 248.809 73.221 0.091 134.767 
(101.484) (186.238) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 


National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 


Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using random effect generalized 


least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 


were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on 


the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the 
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following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. Regression estimates are not shown 
for samples of under 100. “—“ indicates either the sample size was too small or the regression could not be estimated because of collinearity. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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TABLE C.8 


Urban Alliance Program Impacts, GPA 3.0 to 4.0 


Mean ITT TOT 
Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 

Received job help (survey) 

One year 155 0.924 0.853 0.101** 0.321% 0.171* 
(0.052) (0.094) 

Received college help (survey) 

One year 159 0.957 0.899 0.090** 0.346% 0.167** 
(0.040) (0.076) 

Graduated high school (HS data) 

One year 277 1 0.974 — — _ 

Suspended senior year (HS data) 

One year 268 0.044 0.065 -0.021 -0.091 -0.040 
(0.029) (0.062) 

Chronically absent senior year (HS data) 

One year 277 0.206 0.182 0.050 0.126 0.131 
(0.055) (0.106) 

Cumulative GPA (HS data) 

One year 273 3.276 3.222 0.006 0.013 0.013 
(0.051) (0.103) 

Took SAT (survey) 

One year 161 0.957 1.000 -0.036 -0.253 -0.068 
(0.026) (0.049) 

Took ACT (survey) 

One year 161 0.500 0.471 0.009 0.018 0.017 
(0.080) (0.147) 

Filled out FAFSA (survey) 

One year 161 0.979 0.986 0.001 0.006 0.002 
(0.022) (0.042) 

Comfort with FAFSA and scholarships (survey) 

One year 158 3.559 3.418 0.162* 0.276% 0.302* 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(0.096) (0.182) 

Applied to college (survey) 

One year 164 1 0.986 = = - 

Hard skill comfort (survey) 

One year 135 3.650 3.561 0.106 0.151 0.194 
(0.126) (0.230) 

Two years 135 3.763 3.632 0.138 0.232% 0.256 
(0.105) (0.192) 

Soft skill comfort (survey) 

One year 135 3.712 3.646 0.082 0.218% 0.149 
(0.066) (0.122) 

Two years 135 3.813 3.719 0.115 0.271% 0.210 
(0.074) (0.140) 

Goal setting (survey) 

Two years 146 4.250 4.219 0.093 0.088 0.218 
(0.181) (0.338) 

Time management (survey) 

Two years 148 3.780 3.621 0.223 0.214% 0.453 
(0.178) (0.343) 

Attended college (NSC) 

One year 296 0.804 0.791 -0.002 -0.006 -0.011 
(0.049) (0.104) 

Two years 296 0.836 0.826 -0.002 -0.006 -0.012 
(0.045) (0.097) 

Attended four-year college (NSC) 

One year 296 0.687 0.721 -0.072 -0.158 -0.161 
(0.055) (0.115) 

Two years 296 0.724 0.756 -0.062 -0.140 -0.148 
(0.052) (0.110) 

Attended two-year college (NSC) 

One year 296 0.126 0.070 0.077* 0.258% 0.161* 
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Mean ITT TOT 


Observations Regression Standardized IV regression 

Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
(0.041) (0.086) 

Two years 296 0.182 0.116 0.088* 0.248% 0.180* 
(0.048) (0.101) 

Completed one year of college (NSC) 

Two years 296 0.729 0.721 -0.003 -0.008 -0.007 
(0.056) (0.116) 

Completed one year of four-year college (NSC) 

Two years 296 0.636 0.698 -0.095 -0.202 -0.187 
(0.060) (0.122) 

Completed two years of college (NSC) 

Two years 296 0.463 0.465 -0.001 -0.003 0.005 
(0.064) (0.133) 

Completed two years of four-year college (NSC) 

Two years 284 0.435 0.465 -0.043 -0.086 -0.070 
(0.065) (0.130) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 296 0.439 0.419 0.006 0.013 0.020 
(0.063) (0.131) 

Retention rate of college attended (IPEDS) 

One year 214 71.234 69.933 0.855 0.063 1.701 
(1.942) (3.876) 

Two years 217 69.509 68.774 0.409 0.029 1.321 
(2.046) (4.228) 

Graduation rate of college attended (IPEDS) 

One year 212 43.496 43.208 -1.110 -0.052 -2.294 
(3.283) (6.558) 

Two years 217 40.720 42.225 -2.949 -0.128 -5.022 
(3.463) (7.176) 

75th percentile SAT score of college attended (IPEDS) 

One year 149 1,089.404 1,059.738 16.158 0.102 25.761 

(30.070) (56.835) 
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Mean ITT TOT 
Observations Regression Standardized IV regression 
Outcome (data source) (n) Treatment Control adjusted effect size adjusted 
Two years 135 1,088.895 1,067.866 8.390 0.049 21.257 
(31.509) (69.013) 
Held a postprogram job (survey) 
One year 164 0.404 0.486 -0.083 -0.166 -0.173 
(0.077) (0.148) 
Two years 159 0.830 0.857 -0.014 -0.037 -0.032 
(0.059) (0.111) 
Currently employed (survey) 
One year 164 0.330 0.414 -0.092 -0.189 -0.206 
(0.075) (0.146) 
Two years 161 0.468 0.543 -0.044 -0.087 -0.079 
(0.077) (0.147) 
Postprogram wages (survey) 
One year 160 2.209 2.702 -0.535 -0.127 -1.071 
(0.672) (1.263) 
Two years 160 2.878 3.077 -0.228 -0.053 -0.416 
(0.715) (1.328) 
Money accumulated (survey) 
One year 157 419.473 454.511 -78.670 -0.066 -143.930 
(199.973) (366.643) 
Two years 157 373.829 311.675 49.660 0.055 90.855 
(152.014) (277.626) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public Charter School Board, individual charter schools in DC, 
National Student Clearinghouse (NSC), and Integrated Postsecondary Education Data System (IPEDS). 
Notes: IV = instrumental variable; HS = high school. Standard errors are given in parentheses. For continuous outcomes, the results were estimated using random effect generalized 


least squares. For dummy outcomes, the results were estimated using a random effects logit model. Intent to treat (ITT) compared outcomes of a treatment group of individuals who 


were accepted into the program (but who may or may not have completed the internship) with a control group of individuals who were not accepted into the program. Treatment on 
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the treated (TOT) compared outcomes of those in the treatment group who completed the internship to those in the control group. The regression-adjusted models included the 
following control measures: program year, a site dummy, gender, neighborhood percentage poverty, previously held a job, and junior year GPA. Regression estimates are not shown 
for samples of under 100. “—“ indicates either the sample size was too small or the regression could not be estimated because of collinearity. 

*significant at 10% **significant at 5% ***significant at 1% 

“small effect **medium effect **“large effect 
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Appendix D. Full Sample and 
Subgroup Unadjusted Differences at 
One and Two Years 


TABLE D.1 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), Full Sample 


Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
Received job help (survey) 
One year 547 0.948 0.823 0.124*** 
(0.026) 
Received college help (survey) 
One year 549 0.934 0.854 0.079*** 
(0.026) 
Graduated high school (HS data) 
One year 939 0.980 0.958 0.022** 
(0.011) 
Suspended senior year (HS data) 
One year 955 0.077 0.096 -0.019 
(0.019) 
Chronically absent senior year (HS data) 
One year 955 0.322 0.312 0.009 
(0.032) 
Cumulative GPA (HS data) 
One year 926 2.655 2.555 0.101** 
(0.044) 
Took SAT (survey) 
One year 555 0.907 0.911 -0.004 
(0.024) 
Took ACT (survey) 
One year 555 0.443 0.352 0.091** 
(0.041) 
Filled out FAFSA (survey) 
One year 548 0.931 0.921 0.010 
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Mean ITT 


Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 
(0.022) 

Comfort with FAFSA and scholarships (survey) 

One year 541 3.595 3.466 0.129*** 
(0.051) 

Applied to college (survey) 

One year 555 0.942 0.919 0.023 
(0.022) 

Hard skill comfort (survey) 

One year 467 3.656 3.536 0.120* 
(0.062) 

Two years 467 3.656 3.605 0.051 
(0.059) 

Soft skill comfort (survey) 

One year 461 3.731 3.637 0.094*** 
(0.038) 

Two years 461 3.781 3.705 0.076* 
(0.045) 

Goal setting (survey) 

Two years 506 4.160 4.207 -0.047 
(0.099) 

Time management (survey) 

Two years 515 3.708 3.666 0.042 
(0.093) 

Attended college (NSC) 

One year 1,043 0.637 0.583 0.053* 
(0.031) 

Two years 1,043 0.691 0.642 0.049* 
(0.030) 

Attended four-year college (NSC) 

One year 1,043 0.534 0.453 0.081*** 
(0.032) 

Two years 1,043 0.579 0.494 0.085*** 
(0.032) 

Attended two-year college (NSC) 

One year 1,043 0.107 0.133 -0.026 
(0.021) 

Two years 1,043 0.162 0.192 -0.030 
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Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
(0.024) 
Completed one year of college (NSC) 
Two years 1,043 0.522 0.453 0.069** 
(0.032) 
Completed one year of four-year college (NSC) 
Two years 1,043 0.465 0.406 0.059* 
(0.032) 
Completed two years of college (NSC) 
Two years 1,043 0.302 0.244 0.057** 
(0.029) 
Completed two years of four-year college (NSC) 
Two years 1,043 0.286 0.244 0.042 
(0.029) 
Attained two-year degree or enrolled in third year (NSC) 
Two years 1,043 0.278 0.217 0.061** 
(0.028) 
Retention rate of college attended (IPEDS) 
One year 580 65.408 63.239 2.169* 
(1.222) 
Two years 619 65.273 62.980 2.293** 
(1.156) 
Graduation rate of college attended (IPEDS) 
One year 578 35.972 32.543 3.429** 
(1.695) 
Two years 621 34.604 32.657 1.948 
(1.774) 
75th percentile SAT score of college attended (IPEDS) 
One year 315 1,047.124 1,004.250 42.874*** 
(17.251) 
Two years 303 1,044.269 1,015.567 28.702 
(19.085) 
Held a postprogram job (survey) 
One year 561 0.419 0.485 -0.066 
(0.042) 
Two years 555 0.814 0.811 0.002 
(0.033) 
Currently employed (survey) 
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Mean ITT 


Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
One year 561 0.313 0.370 -0.058 
(0.040) 
Two years 555 0.457 0.478 -0.022 
(0.043) 
Postprogram wages (survey) 
Ghevear 559 2.554 2.834 -0.281 
(0.361) 
Tie yeurs 553 2.556 2.559 -0.047 
(0.368) 
Money accumulated (survey) 
One year 554 375.645 337.033 38.612 
(95.009) 
Thaweas 548 311.085 259.511 46.475 
(72.799) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: Standard errors are given in parentheses. 

*significant at 10% **significant at 5% ***significant at 1% 
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TABLE D.2 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), Washington, DC 


Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
Received job help (survey) 
One year 450 0.948 0.822 0.126*** 
(0.029) 
Received college help (survey) 
One year 460 0.928 0.870 0.058** 
(0.028) 
Graduated high school (HS data) 
One year 769 0.977 0.956 0.022* 
(0.013) 
Suspended senior year (HS data) 
One year 784 0.079 0.111 -0.031 
(0.022) 
Chronically absent senior year (HS data) 
One year 784 0.350 0.352 -0.002 
(0.036) 
Cumulative GPA (HS data) 
One year 755 2.735 2.600 0.135*** 
(0.048) 
Took SAT (survey) 
One year 465 0.913 0.908 0.006 
(0.026) 
Took ACT (survey) 
One year 465 0.480 0.410 0.070 
(0.046) 
Filled out FAFSA (survey) 
One year 460 0.925 0.921 0.004 
(0.025) 
Comfort with FAFSA and scholarships (survey) 
One year 453 3.606 3.507 0.099* 
(0.054) 
Applied to college (survey) 
One year 465 0.953 0.931 0.022 
(0.022) 
Hard skill comfort (survey) 
One year 398 3.655 3.573 0.082 
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Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
(0.066) 
Two years 398 3.641 3.640 0.001 
(0.063) 
Soft skill comfort (survey) 
One year 392 3.735 3.635 0.100** 
(0.042) 
Two years 392 3.773 3.709 0.064 
(0.050) 
Goal setting (survey) 
Two years 428 4.129 4.131 -0.002 
(0.112) 
Time management (survey) 
Two years 434 3.677 3.601 0.076 
(0.104) 
Attended college (NSC) 
One year 843 0.667 0.601 0.067* 
(0.035) 
Two years 843 0.726 0.662 0.064** 
(0.033) 
Attended four-year college (NSC) 
One year 843 0.610 0.529 0.082** 
(0.036) 
Two years 843 0.659 0.583 0.076** 
(0.035) 
Attended two-year college (NSC) 
One year 843 0.062 0.076 -0.013 
(0.018) 
Two years 843 0.121 0.119 0.002 
(0.024) 
Completed one year of college (NSC) 
Two years 843 0.571 0.514 0.056 
(0.036) 
Completed one year of four-year college (NSC) 
Two years 843 0.534 0.482 0.052 
(0.036) 
Completed two years of college (NSC) 
Two years 843 0.338 0.291 0.047 
APPENDIX D 99 


Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
(0.034) 
Completed two years of four-year college (NSC) 
Two years 843 0.329 0.291 0.038 
(0.034) 
Attained two-year degree or enrolled in third year (NSC) 
Two years 843 0.310 0.259 0.051 
(0.033) 
Retention rate of college attended (IPEDS) 
One year 509 66.612 64.386 2.226* 
(1.213) 
Two years 550 65.556 63.753 1.803 
(1.245) 
Graduation rate of college attended (IPEDS) 
One year 510 37.700 34.324 3.376** 
(1.718) 
Two years 551 35.626 35.691 -0.065 
(1.849) 
75th percentile SAT score of college attended (IPEDS) 
One year 282 1,050.032 1,001.543 48.489*** 
(19.076) 
Two years 276 1,046.812 1,017.035 29.777 
(20.659) 
Held a postprogram job (survey) 
One year 465 0.413 0.456 -0.043 
(0.046) 
Two years 457 0.823 0.797 0.026 
(0.036) 
Currently employed (survey) 
One year 471 0.311 0.336 -0.025 
(0.043) 
Two years 465 0.457 0.470 -0.012 
(0.047) 
Postprogram wages (survey) 
One year 469 2.496 2.512 -0.016 
(0.391) 
Two years 463 2.665 2.510 0.110 
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Mean ITT 


Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 
(0.408) 

Money accumulated (survey) 

Gneveak 465 383.685 294.270 89.415 
(100.556) 
Tene 459 335.125 219.976 100.043 
(78.866) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: Standard errors are given in parentheses. 

*significant at 10% **significant at 5% ***significant at 1% 
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TABLE D.3 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), Baltimore 


Mean ITT 
Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Received job help (survey) 

One year 89 0.946 0.827 0.119* 
(0.070) 

Received college help (survey) 

One year 89 0.973 0.788 0.185*** 
(0.072) 

Graduated high school (HS data) 

One year 170 0.991 0.967 0.024 
(0.021) 

Suspended senior year (HS data) 

One year 171 0.064 0.033 0.031 
(0.036) 

Chronically absent senior year (HS data) 

One year 171 0.182 0.148 0.034 
(0.060) 

Cumulative GPA (HS data) 

One year 171 2.273 2.380 -0.107 
(0.101) 

Took SAT (survey) 

One year 90 0.865 0.925 -0.060 
(0.065) 

Took ACT (survey) 

One year 90 0.189 0.113 0.076 
(0.076) 

Filled out FAFSA (survey) 

One year 88 0.973 0.922 0.051 
(0.050) 

Comfort with FAFSA and scholarships (survey) 

One year 88 3.514 3.298 0.216 
(0.145) 

Applied to college (survey) 

One year 90 0.865 0.868 -0.003 
(0.074) 

Hard skill comfort (survey) 

One year 69 3.667 3.381 0.286 
(0.195) 

Two years 69 3.778 3.452 0.325* 
(0.168) 

Soft skill comfort (survey) 

One year 69 3.696 3.643 0.053 
(0.092) 

Two years 69 3.852 3.690 0.161 
(0.106) 

Goal setting (survey) 
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Mean ITT 
Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Two years 78 4.429 4.500 -0.071 
(0.182) 

Time management (survey) 

Two years 81 3.961 3.915 0.046 
(0.214) 

Attended college (NSC) 

One year 200 0.487 0.524 -0.037 
(0.072) 

Two years 200 0.521 0.573 -0.052 
(0.072) 

Attended four-year college (NSC) 

One year 200 0.160 0.195 -0.035 
(0.055) 

Two years 200 0.193 0.195 -0.002 
(0.057) 

Attended two-year college (NSC) 

One year 200 0.328 0.329 -0.002 
(0.068) 

Two years 200 0.361 0.439 -0.078 
(0.070) 

Completed one year of college (NSC) 

Two years 200 0.286 0.244 0.042 
(0.064) 

Completed one year of four-year college (NSC) 

Two years 200 0.126 0.146 -0.020 
(0.049) 

Completed two years of college (NSC) 

Two years 200 0.126 0.085 0.041 
(0.045) 

Completed two years of four-year college 

(NSC) 

Two years 200 0.076 0.085 -0.010 
(0.039) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 200 0.118 0.073 0.044 
(0.043) 

Retention rate of college attended (IPEDS) 

One year 71 54.805 57.200 -2.395 
(4.335) 

Two years 69 62.342 58.839 3.503 
(3.155) 

Graduation rate of college attended (IPEDS) 

One year 68 20.374 22.428 -2.055 
(5.405) 

Two years 70 24.280 16.309 7.971 
(5.088) 


75th percentile SAT score of college attended (IPEDS) 
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Mean ITT 


Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
One year 33 1,012.059 1,018.125 -6.066 
(36.117) 
Two years 7 1,013.438 1,004.091 9.347 
(41.678) 
Held a postprogram job (survey) 
One year 90 0.459 0.604 -0.144 
(0.107) 
Two years 90 0.757 0.868 -0.111 
(0.082) 
Currently employed (survey) 
One year 90 0.324 0.509 -0.185* 
(0.106) 
Two years 90 0.459 0.509 -0.040 
(0.117) 
Postprogram wages (survey) 
One year 90 2.948 4.154 -1.206 
(0.944) 
TWovears 90 1.818 2757 -0.425 
(0.925) 
Money accumulated (survey) 
One year 89 321.108 513.019 -191.911 
(278.710) 
TWOweais 89 148.000 422.210 -227.356 
(201.431) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: Standard errors are given in parentheses. 

*significant at 10% **significant at 5% ***significant at 1% 
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TABLE D.4 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), Females 


Mean ITT 
Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Received job help (survey) 

One year 369 0.948 0.844 0.105*** 
(0.031) 

Received college help (survey) 

One year 370 0.948 0.878 0.070** 
(0.029) 

Graduated high school (HS data) 

One year 619 0.977 0.969 0.008 
(0.014) 

Suspended senior year (HS data) 

One year 627 0.066 0.072 -0.006 
(0.022) 

Chronically absent senior year (HS data) 

One year 627 0.363 0.349 0.014 
(0.041) 

Cumulative GPA (HS data) 

One year 615 2.691 2.633 0.058 
(0.055) 

Took SAT (survey) 

One year 373 0.908 0.917 -0.009 
(0.029) 

Took ACT (survey) 

One year 373 0.477 0.387 0.090* 
(0.051) 

Filled out FAFSA (survey) 

One year 371 0.943 0.939 0.004 
(0.024) 

Comfort with FAFSA and scholarships (survey) 

One year 363 3.599 3.471 0.128** 
(0.061) 

Applied to college (survey) 

One year 373 0.933 0.934 -0.000 
(0.026) 

Hard skill comfort (survey) 

One year 321 3.694 3.510 0.184*** 
(0.074) 

Two years 321 3.653 3.621 0.032 
(0.072) 

Soft skill comfort (survey) 

One year 317 3.726 3.616 0.110** 
(0.046) 

Two years 317 3.749 3.737 0.012 
(0.055) 


Goal setting (survey) 


APPENDIX D 


Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
Two years 348 4.182 4.135 0.047 
(0.124) 
Time management (survey) 
Two years 351 3.706 3.619 0.086 
(0.117) 
Attended college (NSC) 
One year 677 0.641 0.634 0.006 
(0.039) 
Two years 677 0.688 0.696 -0.008 
(0.038) 
Attended four-year college (NSC) 
One year 677 0.553 0.511 0.042 
(0.040) 
Two years 677 0.593 0.546 0.046 
(0.040) 
Attended two-year college (NSC) 
One year 677 0.092 0.128 -0.036 
(0.025) 
Two years 677 0.142 0.189 -0.048 
(0.030) 
Completed one year of college (NSC) 
Two years 677 0.542 0.493 0.049 
(0.041) 
Completed one year of four-year college (NSC) 
Two years 677 0.488 0.454 0.034 
(0.041) 
Completed two years of college (NSC) 
Two years 677 0.316 0.295 0.021 
(0.038) 
Completed two years of four-year college (NSC) 
Two years 677 0.296 0.295 0.001 
(0.037) 
Attained two-year degree or enrolled in third year (NSC) 
Two years 677 0.294 0.264 0.030 
(0.037) 
Retention rate of college attended (IPEDS) 
One year 396 65.315 63.697 1.618 
(1.418) 
Two years 414 64.807 62.156 2.651* 
(1.398) 
Graduation rate of college attended (IPEDS) 
One year 394 36.303 33.157 3.146* 
(1.921) 
Two years 415 35.539 33.118 2.421 
(2.065) 
75th percentile SAT score of college attended (IPEDS) 
One year 206 1,039.261 1,010.879 28.382 
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Mean ITT 


Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
(19.718) 
Two years 191 1,039.508 1,000.846 38.662* 
(21.684) 
Held a postprogram job (survey) 
One year 373 0.405 0.436 -0.031 
(0.051) 
Two years 373 0.836 0.807 0.029 
(0.040) 
Currently employed (survey) 
One year 376 0.308 0.326 -0.018 
(0.048) 
Two years 373 0.456 0.492 -0.038 
(0.053) 
Postprogram wages (survey) 
Gneveir 375 2.557 2.557 -0.000 
(0.425) 
onesie 372 2.443 2.520 -0.089 
(0.430) 
Money accumulated (survey) 
Guevedr 372 387.088 288.845 98.243 
(111.473) 
aoenrs 369 275.420 216.856 74.789 
(73.038) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: Standard errors are given in parentheses. 

*significant at 10% **significant at 5% ***significant at 1% 
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TABLE D.5 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), Males 


Mean ITT 
Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Received job help (survey) 

One year 170 0.947 0.782 0.165*** 
(0.049) 

Received college help (survey) 

One year 179 0.904 0.807 0.097* 
(0.052) 

Graduated high school (HS data) 

One year 320 0.986 0.940 0.045** 
(0.020) 

Suspended senior year (HS data) 

One year 328 0.098 0.134 -0.037 
(0.036) 

Chronically absent senior year (HS data) 

One year 328 0.237 0.252 -0.015 
(0.049) 

Cumulative GPA (HS data) 

One year 311 2.581 2.418 0.163** 
(0.073) 

Took SAT (survey) 

One year 174 0.906 0.899 0.007 
(0.044) 

Took ACT (survey) 

One year 182 0.375 0.281 0.094 
(0.069) 

Filled out FAFSA (survey) 

One year 177 0.905 0.882 0.023 
(0.046) 

Comfort with FAFSA and scholarships (survey) 

One year 178 3.585 3.454 0.131 
(0.093) 

Applied to college (survey) 

One year 182 0.958 0.888 0.07 1* 
(0.039) 

Hard skill comfort (survey) 

One year 146 3.575 3.597 -0.022 
(0.115) 

Two years 146 3.663 3.567 0.095 
(0.103) 

Soft skill comfort (survey) 

One year 144 3.743 3.686 0.056 
(0.066) 

Two years 144 3.850 3.631 0.219*** 
(0.078) 
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Mean ITT 
Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Goal setting (survey) 

Two years 158 4.111 4.359 -0.248 
(0.161) 

Time management (survey) 

Two years 164 3.714 3.761 -0.047 
(0.153) 

Attended college (NSC) 

One year 366 0.629 0.496 0.133*** 
(0.053) 

Two years 366 0.696 0.549 0.147*** 
(0.051) 

Attended four-year college (NSC) 

One year 366 0.496 0.353 0.142*** 
(0.053) 

Two years 366 0.554 0.406 0.148*** 
(0.054) 

Attended two-year college (NSC) 

One year 366 0.138 0.143 -0.005 
(0.038) 

Two years 366 0.200 0.195 0.005 
(0.043) 

Completed one year of college (NSC) 

Two years 366 0.483 0.383 0.100* 
(0.054) 

Completed one year of four-year college 

(NSC) 

Two years 366 0.421 0.323 0.098* 
(0.053) 

Completed two years of college (NSC) 

Two years 366 0.275 0.158 0.117*** 
(0.045) 

Completed two years of four-year college (NSC) 

Two years 366 0.267 0.158 0.109** 
(0.045) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 366 0.246 0.135 0.110*** 
(0.043) 

Retention rate of college attended (IPEDS) 

One year 184 65.598 62.161 3.438 
(2.369) 

Two years 205 66.177 64.774 1.403 
(2.054) 

Graduation rate of college attended (IPEDS) 

One year 184 35.290 31.143 4.147 
(3.403) 

Two years 206 32.805 31.644 1.161 
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Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
(3.378) 
75th percentile SAT score of college attended (IPEDS) 
One year 109 1,060.549 987.679 72.870** 
(34.130) 
Two years 112 1,051.852 1,045.469 6.383 
(36.644) 
Held a postprogram job (survey) 
One year 174 0.448 0.584 -0.136* 
(0.073) 
Two years 174 0.771 0.820 -0.049 
(0.060) 
Currently employed (survey) 
One year 185 0.323 0.461 -0.138* 
(0.071) 
Two years 182 0.458 0.449 0.019 
(0.078) 
Postprogram wages (survey) 
One year 184 2.546 3.397 -0.851 
(0.673) 
Twa vests 181 2.787 2.639 0.236 
(0.729) 
Money accumulated (survey) 
One year 182 352.398 436.177 -83.780 
(178.584) 
aneate 179 383.541 347.271 70.314 
(162.960) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 


Education Data System (IPEDS). 


Notes: Standard errors are given in parentheses. 
*significant at 10% **significant at 5% 


significant at 1% 
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TABLE D.6 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), GPA 0.0 to 2.0 


Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
Received job help (survey) 
One year 75 0.939 0.857 0.082 
(0.072) 
Received college help (survey) 
One year 75 0.906 0.791 0.116 
(0.086) 
Graduated high school (HS data) 
One year 129 0.887 0.902 -0.014 
(0.056) 
Suspended senior year (HS data) 
One year 138 0.140 0.130 0.010 
(0.060) 
Chronically absent senior year (HS data) 
One year 138 0.581 0.630 -0.048 
(0.086) 
Cumulative GPA (HS data) 
One year 133 1.731 1.701 0.030 
(0.063) 
Took SAT (survey) 
One year 77 0.765 0.721 0.044 
(0.102) 
Took ACT (survey) 
One year 77 0.353 0.233 0.120 
(0.104) 
Filled out FAFSA (survey) 
One year 74 0.781 0.810 -0.028 
(0.096) 
Comfort with FAFSA and scholarships (survey) 
One year 74 3.545 3.573 -0.028 
(0.144) 
Applied to college (survey) 
One year 77 0.735 0.767 -0.032 
(0.100) 
Hard skill comfort (survey) 
One year 65 3.700 3.629 0.071 
(0.135) 
Two years 65 3.600 3.686 -0.086 
(0.128) 
Soft skill comfort (survey) 
One year 62 3.743 3.729 0.013 
(0.087) 
Two years 62 3.821 3.824 -0.002 
(0.099) 
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Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
Goal setting (survey) 
Two years 72 4.290 4.049 0.242 
(0.276) 
Time management (survey) 
Two years 73 3.672 3.631 0.041 
(0.272) 
Attended college (NSC) 
One year 147 0.315 0.333 -0.018 
(0.079) 
Two years 147 0.391 0.386 0.005 
(0.083) 
Attended four-year college (NSC) 
One year 147 0.196 0.193 0.003 
(0.067) 
Two years 147 0.239 0.246 -0.006 
(0.073) 
Attended two-year college (NSC) 
One year 147 0.120 0.140 -0.021 
(0.057) 
Two years 147 0.174 0.211 -0.037 
(0.066) 
Completed one year of college (NSC) 
Two years 147 0.163 0.175 -0.012 
(0.063) 
Completed one year of four-year college (NSC) 
Two years 147 0.120 0.123 -0.003 
(0.055) 
Completed two years of college (NSC) 
Two years 147 0.098 0.053 0.045 
(0.046) 
Completed two years of four-year college (NSC) 
Two years 147 0.087 0.053 0.034 
(0.044) 
Attained two-year degree or enrolled in third year (NSC) 
Two years 147 0.076 0.035 0.041 
(0.040) 
Retention rate of college attended (IPEDS) 
One year 47 59.655 54.667 4.989 
(3.847) 
Two years 52 59.563 54.550 5.012 
(3.399) 
Graduation rate of college attended (IPEDS) 
One year 48 22.920 24.657 -1.737 
(4.331) 
Two years 52 19.672 20.825 -1.153 
(3.751) 
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Mean ITT 


Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
75th percentile SAT score of college attended (IPEDS) 
One year 8 1,032.000 911.667 120.333 
(73.422) 
Two years 10 1,024.000 944.000 80.000 
(76.220) 
Held a postprogram job (survey) 
One year 77 0.294 0.419 -0.124 
(0.111) 
Two years 77 0.735 0.767 -0.032 
(0.100) 
Currently employed (survey) 
One Year 77 0.206 0.209 -0.003 
(0.094) 
Two Year 77 0.500 0.419 0.040 
(0.125) 
Postprogram wages (survey) 
One year 76 1.985 2.034 -0.049 
(0.823) 
Two years 76 2.156 1.138 1.351* 
(0.824) 
Money accumulated (survey) 
One year 75 341.742 251.119 90.623 
(300.473) 
Two years 75 107.000 32.837 98.065** 
(50.827) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: Standard errors are given in parentheses. 

*significant at 10% **significant at 5% ***significant at 1% 
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TABLE D.7 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), GPA 2.0 to 3.0 


Mean ITT 
Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Received job help (survey) 

One year 259 0.965 0.803 0.162*** 
(0.037) 

Received college help (survey) 

One year 259 0.915 0.866 0.049 
(0.038) 

Graduated high school (HS data) 

One year 482 0.994 0.975 0.019* 
(0.011) 

Suspended senior year (HS data) 

One year 483 0.083 0.093 -0.010 
(0.027) 

Chronically absent senior year (HS data) 

One year 483 0.327 0.253 0.074* 
(0.044) 

Cumulative GPA (HS data) 

One year 474 2.538 2.541 -0.003 
(0.035) 

Took SAT (survey) 

One year 267 0.916 0.953 -0.037 
(0.030) 

Took ACT (survey) 

One year 267 0.441 0.362 0.078 
(0.060) 

Filled out FAFSA (survey) 

One year 264 0.930 0.944 -0.013 
(0.030) 

Comfort with FAFSA and scholarships (survey) 

One year 260 3.612 3.508 0.103 
(0.071) 

Applied to college (survey) 

One year 267 0.951 0.961 -0.010 
(0.025) 

Hard skill comfort (survey) 

One year 229 3.653 3.575 0.078 
(0.086) 

Two years 229 3.605 3.632 -0.027 
(0.087) 

Soft skill comfort (survey) 

One year 226 3.733 3.642 0.091* 
(0.055) 

Two years 226 3.740 3.692 0.048 
(0.070) 
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Mean ITT 
Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Goal setting (survey) 

Two years 242 4.068 4.236 -0.169 
(0.147) 

Time management (survey) 

Two years 247 3.638 3.643 -0.004 
(0.133) 

Attended college (NSC) 

One year 507 0.644 0.603 0.041 
(0.045) 

Two years 507 0.703 0.667 0.036 
(0.043) 

Attended four-year college (NSC) 

One year 507 0.565 0.448 0.116*** 
(0.046) 

Two years 507 0.609 0.494 0.115*** 
(0.046) 

Attended two-year college (NSC) 

One year 507 0.082 0.161 -0.079*** 
(0.029) 

Two years 507 0.144 0.213 -0.069** 
(0.035) 

Completed one year of college (NSC) 

Two years 507 0.518 0.454 0.064 
(0.047) 

Completed one year of four-year college (NSC) 

Two years 507 0.479 0.397 0.083* 
(0.046) 

Completed two years of college (NSC) 

Two years 507 0.268 0.236 0.032 
(0.041) 

Completed two years of four-year college (NSC) 

Two years 507 0.262 0.236 0.026 
(0.041) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 507 0.244 0.218 0.026 
(0.040) 

Retention rate of college attended (IPEDS) 

One year 284 62.221 61.108 1.113 
(1.533) 

Two years 312 63.119 60.816 2.303 
(1.497) 

Graduation rate of college attended (IPEDS) 

One year 283 32.454 28.010 4.444** 
(1.878) 

Two years 313 32.607 29.307 3.300 
(2.169) 
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Mean ITT 


Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
75th percentile SAT score of college attended (IPEDS) 
One year 142 999.207 956.278 42.929** 
(18.968) 
Two years 136 1,002.180 976.159 26.022 
(23.697) 
Held a postprogram job (survey) 
One year 267 0.455 0.496 -0.042 
(0.061) 
Two years 260 0.818 0.811 0.007 
(0.048) 
Currently employed (survey) 
One year 270 0.329 0.394 -0.065 
(0.059) 
Two years 267 0.448 0.449 -0.009 
(0.062) 
Postprogram wages (survey) 
One year 270 2.924 2.942 -0.018 
(0.549) 
Twevess 267 2.424 2.794 -0.225 
(0.553) 
Money accumulated (survey) 
One year 270 371.383 243.111 128.272 
(116.941) 
Two years 267 313.057 —- 248.809 73.221 
(101.484) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: Standard errors are given in parentheses. 

*significant at 10% **significant at 5% ***significant at 1% 
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TABLE D.8 


Urban Alliance Unadjusted Difference in Means (Intent to Treat), GPA 3.0 to 4.0 


Mean ITT 
Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
Received job help (survey) 
One year 155 0.924 0.853 0.071 
(0.049) 
Received college help (survey) 
One year 159 0.957 0.899 0.058 
(0.040) 
Graduated high school (HS data) 
One year 277 1.000 0.974 0.026** 
(0.011) 
Suspended senior year (HS data) 
One year 268 0.044 0.065 -0.021 
(0.029) 
Chronically absent senior year (HS data) 
One year 277 0.206 0.182 0.024 
(0.054) 
Cumulative GPA (HS data) 
One year 273 3.276 3.222 0.054 
(0.061) 
Took SAT (survey) 
One year 161 0.957 1.000 -0.043* 
(0.024) 
Took ACT (survey) 
One year 161 0.500 0.471 0.029 
(0.079) 
Filled out FAFSA (survey) 
One year 161 0.979 0.986 -0.007 
(0.021) 
Comfort with FAFSA and scholarships (survey) 
One year 158 3.559 3.418 0.141 
(0.093) 
Applied to college (survey) 
One year 164 1.000 0.986 0.014 
(0.012) 
Hard skill comfort (survey) 
One year 135 3.650 3.561 0.089 
(0.121) 
Two years 135 3.763 3.632 0.131 
(0.101) 
Soft skill comfort (survey) 
One year 135 3.712 3.646 0.067 
(0.064) 
Two years 135 3.813 3.719 0.093 
(0.073) 
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Mean ITT 


Observations Unadjusted 

Outcome (data source) (n) Treatment Control difference 

Goal setting (survey) 

Two years 146 4.250 4.219 0.031 
(0.176) 

Time management (survey) 

Two years 148 3.780 3.621 0.160 
(0.172) 

Attended college (NSC) 

One year 296 0.804 0.791 0.013 
(0.051) 

Two years 296 0.836 0.826 0.011 
(0.048) 

Attended four-year college (NSC) 

One year 296 0.687 0.721 -0.034 
(0.059) 

Two years 296 0.724 0.756 -0.032 
(0.057) 

Attended two-year college (NSC) 

One year 296 0.126 0.070 0.056 
(0.040) 

Two years 296 0.182 0.116 0.066 
(0.047) 

Completed one year of college (NSC) 

Two years 296 0.729 0.721 0.008 
(0.057) 

Completed one year of four-year college (NSC) 

Two years 296 0.636 0.698 -0.062 
(0.061) 

Completed two years of college (NSC) 

Two years 296 0.463 0.465 -0.002 
(0.064) 

Completed two years of four-year college (NSC) 

Two years 284 0.435 0.465 -0.031 
(0.064) 

Attained two-year degree or enrolled in third year (NSC) 

Two years 296 0.439 0.419 0.021 
(0.063) 

Retention rate of college attended (IPEDS) 

One year 214 71.234 69.933 1.301 
(2.090) 
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Mean ITT 


Observations Unadjusted 
Outcome (data source) (n) Treatment Control difference 
Two years 217 69.509 68.774 0.735 
(2.101) 
Graduation rate of college attended (IPEDS) 
One year 212 43.496 43.208 0.288 
(3.255) 
Two years 217 40.720 42.225 -1.505 
(3.441) 
75th percentile SAT score of college attended (IPEDS) 
One year 149 1,089.404 1,059.738 29.666 
(28.824) 
Two years 135 1,088.895 1,067.866 21.029 
(31.983) 
Held a postprogram job (survey) 
One year 159 0.404 0.486 -0.081 
(0.079) 
Two years 159 0.830 0.857 -0.027 
(0.058) 
Currently employed (survey) 
One year 164 0.330 0.414 -0.084 
(0.076) 
Two years 161 0.468 0.543 -0.044 
(0.080) 
Postprogram wages (survey) 
Gnevear 163 2.209 2.702 -0.493 
(0.660) 
Tuesears 160 2.878 3.077 -0.228 
(0.715) 
Money accumulated (survey) 
One year 160 419.473 454.511 -35.038 
(193.111) 
Tae 157 373.829 311.675 49.660 
(152.014) 


Sources: Urban Alliance program data, final outcome survey, DC Public Schools, Baltimore Public School Board, DC Public 
Charter School Board, individual charter schools in DC, National Student Clearinghouse (NSC), and Integrated Postsecondary 
Education Data System (IPEDS). 

Notes: Standard errors are given in parentheses. 

*significant at 10% **significant at 5% ***significant at 1% 
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Appendix E. Outcomes Codebook 


Table E.1 provides further descriptions of our reported outcomes. An outcome codebook for the one- 


year outcomes can be found in the interim report (Theodos, Pergamit, Hanson, et al. 2016). 


TABLE E.1 


Outcome Measures Codebook 


Outcome Data Source Description 


Comfort level performing general office work, such as using 
Excel, making photocopies, or filing papers; 1-4, higher 
Hard skill comfort Survey value indicates greater comfort 


Urban Institute-developed scale 
Average comfort level speaking with adult coworkers, 
writing professional e-mails, making a presentation, 
dressing professionally, completing work assignments on 
time, and getting to work on time; 
1-4, higher value indicates greater comfort; 

Soft skill comfort Survey Cronbach alpha = 0.79 


Urban Institute -developed scale 
Average amount respondents attribute the following 
statements to themselves: 
| set goals for myself 
| find ways to achieve my goals 
| consider possible obstacles when making plans 
1-5, higher values indicate more "like me"; 
Goal setting Survey Cronbach alpha = 0.91 


Urban Institute -developed scale 

Average amount respondents attribute the following 
statements to themselves: 

| organize my time and do not procrastinate (not put things 
off) 

| set appropriate priorities 

| practice self-discipline 

1-5, higher values indicate more "like me"; 


Time management Survey Cronbach alpha = 0.77 

Attended college NSC Attended college as of fall 2015 

Attended four-year college NSC Attended a four-year college as of fall 2015 
Attended two-year college NSC Attended a two-year college as of fall 2015 
Completed one year of college NSC As of fall 2015 

Completed one year of four- 

year college NSC As of fall 2015 

Completed one year of two- 

year college NSC As of fall 2015 

Completed two years of college NSC As of fall 2015 
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Outcome Data Source Description 
Completed two years of four- 
year college NSC As of fall 2015 
Completed two years of two- 
year college NSC As of fall 2015 
Completed two years of college 
and still enrolled or attained a 
two-year degree NSC As of fall 2015 
The full-time retention rate is the percentage of the (fall 
full-time cohort from the prior year minus exclusions from 
Retention rate of college the fall full-time cohort) that re-enrolled at the institution 
attended IPEDS as either full- or part-time in the current year. 
Graduation rate of college Six-year graduation rate for four-year institutions and 
attended IPEDS three-year graduation rate for two-year institutions 
75th percentile SAT score of Sum of 75th percentile SAT math and verbal scores of 
college attended IPEDS accepted students 
Held a postprogram job Survey Student held at least one job after the program period 
Currently employed Survey Student held at least one job at the time of the survey 
Postprogram wages Survey Average wages earned after the program period 
Money accumulated Survey Current balance in checking, savings, and other accounts 
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Appendix F. Subgroup 
Characteristics 


TABLE F.1 


Characteristics of Urban Alliance Applicants, by Treatment Status and Site 


Washington, DC Baltimore 
Level of Level of 

Characteristic Treatment Control significance Treatment Control significance 
Share in treatment 67% 59% 
Share completed 41% 42% 
Demographic 
characteristics 
Female 67% 62% 61% 67% 
US citizen 94% 97% 96% 99% 
English language 
learner 13% 14% 1% 2% 
Race and ethnicity 
African American 88% 91% 90% 91% 
Hispanic 6% 5% 3% 0% 
Other 4% 4% 3% 2% 
White 1% 0% 3% 6% 
Family characteristics 
Has a child 4% 4% 6% 3% 
Employed adult in 
household 77% 77% 75% 83% 
Living Arrangement 
Mother only 5% 4% 4% 8% 
Father only 55% 59% 61% 49% 
Other 11% 13% 11% 16% 
Two parents 28% 24% 24% 27% 
Other characteristics 
Had a previous job 77% 73% 71% 80% 
Has a checking or 
savings account 35% 43% = 36% 40% 
Money saved 56% 86% 283% 185% 
Observations (n) 580 280 119 82 


Source: Urban Alliance High School Internship Program application forms. 
Notes: All items had a response rate of 80 percent or more except bank account (71 percent). 
*significant at 10% **significant at 5% ***significant at 1% 
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TABLE F.2 


Characteristics of Urban Alliance Applicants, by Treatment Status and Gender 


Female Male 
Level of Level of 

Characteristic Treatment Control — significance Treatment Control — significance 
Share in treatment 67% 64% 
Share completed 42% 39% 
Demographic 
characteristics 
Female 100% 100% 0% 0% 
US citizen 94% 97% 96% 98% 
English language learner 11% 9% 12% 14% 
Race and ethnicity 
African American 89% 93% 88% 87% 
Hispanic 6% 4% 5% 5% 
Other 4% 2% 5% 5% 
White 1% 1% 3% 2% 
Family characteristics 
Has a child 5% 4% 3% 2% 
Employed adult in 
household 78% 77% 75% 81% 
Living arrangement 
Mother only 4% 3% 7% 9% 
Father only 57% 62% 54% 47% 
Other 11% 12% 11% 16% 
Two parents 27% 23% 28% 28% 
Other characteristics 
Had a previous job 78% 77% 72% 69% 
Has a checking or savings 
account 34% 43% i 38% 42% 
Money saved 59% 117% 161% 92% 
Observations (n) 459 228 240 134 


Source: Urban Alliance High School Internship Program application forms. 
Notes: All items had a response rate of 80 percent or more except bank account (71 percent). 
*significant at 10% **significant at 5% ***significant at 1% 
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TABLE F.3 
Characteristics of Urban Alliance Applicants, by Treatment Status and GPA 


GPA 0.0-2.0 GPA 2.0-3.0 GPA 3.0-4.0 
Level of Level of Level of 

Characteristic Treatment Control — significance Treatment Control — significance Treatment Control significance 
Share in treatment 61% 66% 71% 
Share completed 22% 46% 44% 
Demographic 
characteristics 
Female 61% 55% 66% 60% 70% 74% 
US citizen 96% 100% 98% 96% 88% 96% ** 
English language 
learner 13% 6% 8% 12% 16% 16% 
Race and ethnicity 
African American 87% 86% 90% 93% 86% 88% 
Hispanic 8% 7% 5% 4% 7% 3% 
Other 3% 5% 5% 3% 5% 5% 
White 2% 2% 1% 1% 3% 3% 
Family characteristics 
Has a child 8% 10% 5% 2% 2% 0% 
Employed adult in 
household 71% 77% 78% 77% 79% 80% 
Living arrangement 
Father only 6% 4% 5% 6% 4% 3% 
Mother only 64% 67% 56% 57% 52% 49% 
Other 11% 13% 12% 13% 9% 11% 
Two parents 19% 15% 27% 24% 34% 37% 
Other 
characteristics 
Had a previous job 69% 69% 75% 73% 81% 77% 
Has a checking or 
savings account 30% 41% 32% 39% 41% 47% 
Money saved 44% 31% 55% 29% 169% 152% 
Observations (n) 92 58 340 175 214 86 


Source: Urban Alliance High School Internship Program application forms. 
Notes: All items had a response rate of 80 percent or more except bank account (71 percent). 
*significant at 10% **significant at 5% ***significant at 1% 
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Appendix G. Survey Instrument 


This appendix shows the Urban Alliance evaluation final survey (cohort 2, Web version). The Urban 
Alliance evaluation interim survey can be found in the interim report (Theodos, Pergamit, Hanson, et al. 


2016). 


Questions 1 through 4 confirmed that the respondent was speaking and that this was an acceptable 


time to take the survey. 


(ASK W1 NONRESPONDENTS OR #M1029 QN4=3 ) 
5, Do you have either a high school diploma or GED? 


(Please select only one answer) 


1 Highschool diploma 

2 GED 

3 Neither 

X — Blank/No Answer 
(ASK Q6 IF Q5=3) 


6. Are you taking additional courses to obtain your GED? 
1 Yes 
2 No 


X — Blank/No Answer 
(NEW SCREEN) 


(ASK W1 NONRESPONDENTS OR #M1029 QN4=3) 


7. When you were in high school, did you take any Advanced Placement or IB classes to earn 
credit for college? 


(IB stands for International Baccalaureate) 


1 Yes 
2 No 
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8 Don’t Know 
X — Blank/No Answer 


(ASK W1 NONRESPONDENTS OR #M1029 QN4=3) 


8. When you were in high school, did you take any college classes for credit? 
1 Yes 
2 No 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK W1 NONRESPONDENTS OR #M1029 QN4=3) 
9. Did you take either the SAT or ACT? 


SAT 

ACT 

Both 

Neither 

Blank/No Answer 


<x FR WYN RB 


(NEW SCREEN) 
(ASK Q11 IF Q9=1, 3) 
(IF RESPONDENT TOOK THE SAT) 


11. | Please enter your total SAT score: 


(PN: ACCEPT ANSWER 0-2400) 
8 Don’t Know 
X — Blank/No Answer 


(ASK Q11A IF Q11=8) 
11A. Was your score... 


1 1800 or better 
2 1600to1790 
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3 1450to 1590 

4 1250to 1440 

5 Less than 1250 

8 Don’t Know 

X — Blank/No Answer 
(NEW SCREEN) 


(ASK Q12 IF Q9=2, 3) 
(IF RESPONDENT TOOK THE ACT) 


12. Please enter your composite ACT score: 


(PN: ACCEPT ANSWER 0-36) 
8 Don’t Know 
X — Blank/No Answer 


(ASK Q12A IF Q12=8) 
12A. Was your score... 


26 or higher 
23 to 25 
20 to 22 
16to 19 
Less than 16 
Don’t Know 


Blank/No Answer 


x oun BF WN RB 


(NEW SCREEN) 
(ASK W1 NONRESPONDENTS OR #M1029 QN4=3) 
13. Inorder to pay for education after high school did you complete the FAFSA? 


(The FAFSA is the Free Application for Federal Student Aid) 
1 Yes 
No 


8 Don’t Know 
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X — Blank/No Answer 


(ASK W1 NONRESPONDENTS OR #M1029 QN4=3) 


14. Inorder to pay for education after high school did you apply for grants or scholarships? 


1 Yes 

2 No 

8 Don’t Know 

X — Blank/No Answer 
(NEW SCREEN) 


(READ IF W1 NONRESPONDENTS OR #M1029 QN4=3) 


High school students sometimes attend classes or workshops outside of regular school to learn about 
educational opportunities or develop new job skills. These programs might be offered by a high school 
or college, a non-profit such as the Urban Alliance, or a local business. 


(ASK W1 NONRESPONDENTS OR #M1029 QN4=3) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-g) SHOULD BE ON THE LEFT) 
23. Whenyou were a high school senior, did you ever attend a class or workshop where you... 


1 Yes 
2 No 
8 Don’t Know 
X — Blank/No Answer 
a. Received help choosing a college, such as requesting brochures, writing an admissions essay, 


or applying for admission 


b. Learned about options for paying for college, including completing a FAFSA or applying for 
scholarships 


C. Received career counseling or advice 

d. Learned how to get a job, including creating a resume, writing a cover letter, or completing 
applications 

e. Developed general office skills, such as learning how to use Excel, make photocopies, or file 
papers 

f. Developed communication skills, such as speaking with co-workers, making a presentation, or 


composing a professional email 


g. Learned how to behave on a job, such as how to dress or manage your time 
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(NEW SCREEN) 

(ASK Q24 IF Q23A-G=1) 

(PN: SET UP ASA FLEXIBLE GRID. ITEMS (a-g) SHOULD BE ON THE LEFT) 

(PN: IF Q23=1 FOR MORE THAN ONE ITEM IN Q23 INSERT SECOND VERBIAGE IN PARENS) 


24. Who offered (the class or workshop you attended/these classes or workshops you attended)? 


Was it... 

1 Yes 

2 No 

8 Don’t Know 

X — Blank/No Answer 


a. A high school (This includes your own high school as long as the classes or workshops were 
outside regular school hours) 


b. A local college 
Cc Urban Alliance 
d. Another non-profit organization 


A local business 


sh 


A city or government program 


g. Other (SPECIFY) 


(NEW SCREEN) 
(ASK Q25 IF Q23A-G=1) 
(PN: IF Q23=1 FOR MORE THAN ONE ITEM IN Q23 INSERT SECOND VERBIAGE IN PARENS) 


25. |About how many total hours did you spend in (this class or workshop/these classes or 
workshops)? Would you say it was... 


(Your best estimate is fine) 


Under 10 hours 
10 to 25 hours 
26 to 50 hours 
51 to 75 hours 
76 to 100 hours 
Over 100 hours 


nn BR WN PB 
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8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 

(ASK W1 NONRESPONDENTS ONLY) 

(INSERT RESPONSES 5, 6, 10 IF TREATMENT GROUP) 

(PN: ALLOW MULTIPLE RESPONSES, EXCEPT FOR RESPONSE 11) 
(PN: IF Q26=11, ONLY ALLOW SINGLE RESPONSE) 


26. Thinking about all the help you’ve received preparing for your future education, who 
provided this help? Was it... 


(Please select all that apply) 


1. Parent or Foster Parent 

2 Other Relative 

3. Friend or Acquaintance 

4  Anemployer or co-worker (includes current and former) 

5 (ONLY FOR TREATMENT GROUP) Urban Alliance Program Coordinator (“PC”) 
6 (ONLY FOR TREATMENT GROUP) Urban Alliance Alumni Services Staff 
7 Caseworker 

8 Teacher, school counselor, or coach 

9 ~~ Clergyperson 

10 (ONLY FOR TREATMENT GROUP) Other Urban Alliance Staff 

11 Noone 

12 Other 

X — Blank/No Answer 


(NEW SCREEN) 

(ASK W1 NONRESPONDENTS ONLY) 

(INSERT RESPONSES 5, 6, 10 IF TREATMENT GROUP) 

(PN: ALLOW MULTIPLE RESPONSES EXCEPT FOR RESPONSE 11) 
(PN: IF Q27=11, ONLY ALLOW SINGLE RESPONSE) 


27. Thinking about all the help you’ve gotten preparing to get and keep a job, who provided this 
help? Was it... 
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(Please select all that apply) 


1. Parent or Foster Parent 

2 Other Relative 

3. Friend or Acquaintance 

4 Anemployer or co-worker (includes current and former) 

5 (ONLY FOR TREATMENT GROUP) Urban Alliance Program Coordinator (“PC”) 
6 (ONLY FOR TREATMENT GROUP) Urban Alliance Alumni Services Staff 
7 Caseworker 

8 Teacher, school counselor, or coach 

9  Clergyperson 

10 (ONLY FOR TREATMENT GROUP) Other Urban Alliance Staff 

11 Noone 

12 Other 

X — Blank/No Answer 


(NEW SCREEN) 
(ASK Q28-Q33 IF TREATMENT GROUP AND W1 NONRESPONDENT) 
28. Youapplied for the program at Urban Alliance. Did you attend any pre-work training? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK Q29 IF Q28=2) 


(PN: ALLOW MULTIPLE RESPONSES) 
29. Why didn’t you attend any pre-work training? 


(Please select all that apply) 


Class schedule changed 


Extra-curricular activities conflicted 


1 

2 

3 Nolonger interested 
4 Parents wouldn't let me 
5 


Other (SPECIFY) 
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8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK Q30 IF Q28=1) 


30. Did you complete pre-work training? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK Q31 IF Q30=2) 
31. Whydidn’t you complete pre-work training? 


(Please select all that apply) 


Class schedule changed 
Extra-curricular activities conflicted 
No longer interested 

Parents wouldn't let me 

Other (SPECIFY) 

Don’t Know 


Blank/No Answer 


x oun BF WY BR 


(NEW SCREEN) 
(ASK IF TREATMENT GROUP AND W1 NONRESPONDENT) 


32. Did youcomplete an internship with Urban Alliance? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK Q33 IF Q32=2) 
(PN: ALLOW MULTIPLE RESPONSES) 
(SCRAMBLE 1-9) 
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33. | Why didn’t you complete your internship with Urban Alliance? 


(Please select all that apply) 


Never got assigned to ajob 

Job was boring 

Money wasn’t enough 

Change in class schedule 

They asked me to leave 

Class schedule changed 
Extra-curricular activities conflicted 


No longer interested 


© ON WO WT FF WN BF 


Parents wouldn't let me 
Other (SPECIFY) 
Blank/No Answer 


* 6 


[TOPIC: CURRENT EDUCATION] 
INSERT: Let’s talk about your current education. 


(ASK IF #M1029 QN40=1 OR #M1029 QN58=1 - ATTENDING INSTITUTION/EDUCATION 
PROGRAM AT W1) 


(INSERT “INSTITUTION/PROGRAM” FROM #M1029 QN37 IF Q40=1 OR FROM #M1029 QN55 IF 
QN58=1) 


(PN: IF COLLEGE NAME NOT AVAILABLE, INSERT “college” IF QN40=1 OR “a vocational, technical, 
training, or trade program” IF QN58=1 FOR FIRST PARENS) 


(PN: IF COLLEGE NAME NOT AVAILABLE, INSERT “this college” IF QN40=1 OR “this vocational, 
technical, training, or trade program” IF QN58=1 FOR SECOND PARENS) 


24w. At the last survey, you said you were attending [INSERT W1 INSTITUTION/PROGRAM]. Are 
you currently attending [INSERT W1 INSTITUTION/PROGRAM]? 


1 Yes 


2 No 
X — Blank/No Answer 
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(ASK IF Q24w=2) 
(PN: ALLOW BOTH MONTH AND YEAR) 
41. Whendid you stop? Please enter the month and year. 


(PN: ACCEPT 1-12/2013-2015) 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK Q42 IF Q24w=2) 
INSERT “INSTITUTION/PROGRAM” FROM #M1029 QN37 or QN55) 


( 

(PN: IF COLLEGE NAME NOT AVAILABLE, INSERT “this college” IF QN40=1 OR “this vocational, 
technical, training, or trade program” IF QN58=1 FOR PARENS) 
( 


PN: ALLOW ONLY ONE RESPONSE) 
42. What would you say is the main reason that you left [INSERT W1 INSTITUTION/PROGRAM]? 


(Please select only one response) 


Transferred to a better program or four-year college 
Received degree or completed course work 

Offered ajob 

Financial difficulties or couldn’t afford to go 

Did not like school or did not get along with other students 
Poor grades or failed 

Entered military 


Moved away from school 


So ON WO WN FPF WN BF 


Got married 


BR 
oO 


Pregnant, or became the father/mother of a baby 


a= 
= 


Other child care responsibilities 


12 Other family responsibilities 


as 
wo 


Personal health or substance problems 
Other (Specify) 
Blank/No Answer 


oe 


(ASK Q27 IF Q24w=2) 
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(INSERT “INSTITUTION/PROGRAM” FROM #M1029 QN37 OR QN55) 


(PN: IF COLLEGE NAME NOT AVAILABLE, INSERT “this college” IF QN40=1 OR “this vocational, 
technical, training, or trade program” IF QN58=1 FOR PARENS) 


27w. Have you attended a post high school education program or institution since attending [W1 
INSTITUTION/PROGRAM]? 


1 Yes 
2 No 
xX — Blank/No Answer 


(ASK IF #M1029 QN35=2 AND #M1029 QN53=2 OR W1 NONRESPONDENT) 


(IF #M1029 QN35=2 AND QN53=2, INSERT FIRST VERBIAGE; IF W1 NONRESPONDENT, INSERT 
SECOND VERBIAGE) 


28w. At the last survey, you said you were not attending a post high school education program or 
institution. Have you attended a post high school education program or institution since [W1 
Survey Date]?/Have you ever attended a post high school education program or institution? 


1 Yes 
2 No 
xX — Blank/No Answer 


(ASK IF 27w=1 OR 28w=1) 


(PN: IF #$M1029 QN35=1 AND #M1029 QN40=1, INSERT COLLEGE NAME FROM QN37; IF #M1029 
QN53=1 AND #M1029 QN58=1, INSERT PROGRAM NAME FROM QN55;) 


(PN: INSERT SURVEY DATE FOR W1 RESPONDENTS WITH COLLEGE NAME NOT LISTED; INSERT 
SECOND VERBIAGE FOR W1 NONRESPONDENTS) 


29w. Please enter the number of post high school education programs or institutions you have 
attended since [attending INSERT NAME OF W11 INSTITUTION/EDUCATION 
PROGRAM]/W 11 SURVEY DATE/June 2013]: 


(PN: ACCEPT ANSWERS 1-10) 
X — Blank/No Answer 


NEW SCREEN) 
ASK Q37-46, 51, 52 IF Q29w >=1) 
ASK Q37-46 FOR EACH NUMBER OF COLLEGES SPECIFIED IN Q29w) 


( 
( 
( 
(PN: IF Q29w =1, INSERT FIRST VERBIAGE; IF Q29w >1, INSERT SECOND VERBIAGE) 
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37. Please enter the name of this program or institution: / Please enter the name of the program 
or institution you are currently attending, or the program or institution you attended most 
recently: ...Please enter the name of the program or institution you attended prior to the last 
program or institution you mentioned: 


(Please enter response below by providing the full name of the college) 


(Name of program or institution) 


X — Blank/No Answer 


(ASK Q31 IF Q29w >=1) 
31w. What kind of institution/program is (INSERT Q37 WAVE 2 INSTITUTION/PROGRAM 


NAME)? 

1. Technical, vocational 
2 Two-year college 

3 Four-year college 

X — Blank/No Answer 


(ASK IF Q29w >=1) 
(PN: ALLOW BOTH MONTH AND YEAR) 


39. When did you start attending (INSERT Q37 WAVE 2 INSTITUTION/PROGRAM NAME)? 
Please enter the month and year. 


(PN: ACCEPT 1-12/2013-2015) 
X — Blank/No Answer 


(ASK IF Q29w >=1) 
(NEW SCREEN) 
40. Areyoustill attending (INSERT Q37 WAVE 2 INSTITUTION/PROGRAM NAME)? 


1 Yes 
2 No 
X — Blank/No Answer 
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(ASK Q41 IF Q40=2) 
(PN: ALLOW BOTH MONTH AND YEAR) 
41a. Whendid you stop? Please enter the month and year. 


(PN: ACCEPT 1-12/2013-2015) 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK Q42 IF Q40=2) 
(PN: ALLOW ONLY ONE RESPONSE) 


42. What would you say is the main reason that you left this institution or program? 


(Please select only one response) 


Transferred to a better program or four-year college 
Received degree or completed course work 

Offered ajob 

Financial difficulties or couldn’t afford to go 

Did not like school or did not get along with other students 
Poor grades or failed 

Entered military 


Moved away from school 


oO ON WO WN FPF WN BF 


Got married 


BR 
oO 


Pregnant, or became the father/mother of a baby 


a= 
a= 


Other child care responsibilities 


12 Other family responsibilities 


as 
wo 


Personal health or substance problems 
Other (Specify) 
Blank/No Answer 


eee 


(NEW SCREEN) 
(ASK IF Q24w =1 OR Q40=1) 


43. Areyoua full-time or part-time student? 
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1 Full-time student 
2. Part-time student 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK IF Q24w =1 OR Q40=1) 


A4, 


How many credits have you earned at this school? Include credits applied from high school 
and credits from all complete courses. Your best estimate is fine. 


(Please enter response below) 


___ (PN: ACCEPT ANSWERS 0-200) 

1 | have not completed a full semester yet 
8 Don’t Know 

X — Blank/No Answer 


(ASK IF Q24w =1 OR Q40=1) 


45. Please enter the total number of credits required to graduate. Your best estimate is fine. 
(PN: ACCEPT ANSWERS 0-200) 
2  Notapplicable 
8 Don’t Know 
X — Blank/No Answer 
(NEW SCREEN) 


(ASK IF Q24w =1 OR Q40=1) 
(PN: ALLOW ONE DECIMAL) 


46. 


What is your total GPA across all terms at (INSERT Q37 WAVE 2 INSTITUTION/PROGRAM 
NAME)? 


Your best estimate is fine. 


(Please enter response below) 


(PN: ACCEPT ANSWERS 0-4.50) 


1 | have not completed a full semester yet 
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8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


51. Have you received a certificate, license, or degree? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK Q52 IF Q51=1) 
(PN: ALLOW MULTIPLE RESPONSES) 


Qn51. Please enter the types of certificates, licenses, or degrees you have received: 


______ (Specify) 
____ (Specify) 
_____ (Specify) 
8 Don’t Know 
X — Blank/No Answer 


[TOPIC: TRANSFER/PERSISTENCE] 


(ASK Q42w IF Q24w= 1 OR Q40 = 1) 


42w. Do you plan to transfer from (INSERT Q37 WAVE 2 INSTITUTION/PROGRAM NAME) to 
another program or institution? 


(Please enter response below) 


Yes 
No 
Don’t know 


Blank/No Answer 


Xx ©O NM 


(ASK IF Q42w = 1) 
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43w. Why do you plan to transfer? 


1. Totransfer to abetter program or to a four-year college 
2 Moved away from school 
3. Financial difficulties or cannot afford to go to this institution 
4  Donot like school or do not get along with other students 
5 Poor grades or failing at institution 
6 Entered or entering military 
7 Pregnant, or became the father/mother of a baby 
8 = Other child care responsibilities 
9 Other family responsibilities 
10 Personal health or substance problems 
11 Other (Specify) 
X — Blank/No Answer 
(NEW SCREEN) 


(ASK Q50 IF Q31w=2 FOR ANY COLLEGE LISTED) 
(ASK ONLY RESPONDENTS WHO ATTENDED A TWO-YEAR COLLEGE) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-d) SHOULD BE ON THE LEFT) 


50. The following is a list of reasons why people might enroll in a two-year college. For each one, 
please tell me if it applies to you. 


1 Yes 
2 No 
X — Blank/No Answer 


a. To obtain or maintain skills for a current or future job 
b. To obtain or maintain a license or certification 
C. To take courses before transferring to a four-year college 
d. To obtain a certificate or an Associate’s Degree 
(NEW SCREEN) 


(PN: ASK IF (W1 NONRESPONDENTS AND 28w =1) OR (#M1029 QN4=3 and #M1029 qn35 = 1) 


47. Since you began college, have you ever taken a remedial, adult basic education (ABE), or 
developmental learning course? These courses help students develop basic reading, writing, 
and mathematic skills to be successful in college. 
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1 Yes 
2 No 
X — Blank/No Answer 


(ASK Q48 IF Q47=1) 


48. Please enter the number of remedial, adult basic education (ABE) or developmental learning 
courses you have taken: 


___ (PN: ACCEPT ASNWER 1-50) 
X — Blank/No Answer 


(NEW SCREEN) 


(ASK IF #M1029 QN35=1 OR #M1029 QN53=1 OR Q29w>=1 - IF ATTENDED 
INSTITUTION/PROGRAM) 


(INSERT Q49C ONLY IF RESPONDENT ATTENDED HIGH SCHOOL IN DC) 
(PN: INSERT FIRST RESPONSE IN Q37, THE NAME OF THE MOST RECENT COLLEGE) 


(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-h) SHOULD BE ON THE LEFT; PERCENTAGES ENTERED 
SHOULD ADD UP TO NO MORE THAN 100%) 


(PN: IF #M1029 QN35=1 AND #M1029 QN40=1, INSERT COLLEGE NAME FROM QN37; IF #M1029 
QN53=1 AND #M1029 QN58=1, INSERT PROGRAM NAME FROM QN55; IF Q40_01=1, INSERT 
NAME FROM Q37) 


(PN: IF COLLEGE NAME NOT AVAILABLE, INSERT “college” IF QN40=1 OR “the vocational, technical, 
training, or trade program” IF QN58=1 FOR FIRST PARENS) 


49. The following is a list of ways that people pay for school. For each one, tell me the percentage 
of your expenses while attending (INSERT W1 RESPONSE/INSERT W2 RESPONSE) that is 
covered by each item. 


(If one of the following items do not help cover your expenses, please enter “O”) 
1 (Please enter a percentage 0-100%) 
8 Don’t Know 
X — Blank/No Answer 
a. Parents/spouse/relatives 


b. Work-study program 
GC; TAG/LEAP (DC ONLY) 
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d. Other Grants/scholarships 


e. Student loans, other loans (e.g., bank) 


sh 


Employer contribution program 


Public assistance (e.g., welfare, unemployment) 


> 9 


Other personal income/savings 


(PN: IF q49a-h DO NOT EQUAL 100%, PLEASE ADD THE FOLLOWING ERROR MESSAGE: “Please use 
the counter below to help you add your expenses to 100%.) 


[TOPIC: MILITARY SERVICE] 
(ASK ALL) 


48w. Are you currently enlisted in the military? 


1 Yes 
2 No 
xX — Blank/No Answer 


[TOPIC: SKILLS] 


(NEW SCREEN) 
(ASK ALL) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-k) SHOULD BE ON THE LEFT) 


34. For each of the following activities, please indicate whether today you would feel very 
comfortable, somewhat comfortable, somewhat uncomfortable, or very uncomfortable in 
completing the activity. 


Very Comfortable 
Somewhat Comfortable 
Somewhat UNcomfortable 
Very UNcomfortable 
Don’t Know 


Blank/No Answer 


xX wo FR WOW NY BF 


a. Identifying grants and scholarships to help pay for college 
b. Completing the FAFSA or scholarship applications 
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cw. Applying for post-high school education 

dw. Figuring out your education and career goals 

ew. Finding jobs, internships, or other employment 
Writing a cover letter or resume 


Cc 
d. Completing a job application 


e. Asking someone to serve as a job reference 

f. Being interviewed for a job 

g Performing general office work, such as using Excel, making photocopies, or filing papers 
h. Speaking with adult co-workers and writing professional emails 


i. Making a presentation 

j. Dressing professionally 

k. Completing work assignments on time 
I. Getting to work on time 

m. — Receiving and dealing with criticism 


n. Approaching and speaking with working professionals (networking) 


(NEW SCREEN) 


(ASK IF #M1029 QN35=1 OR #M1029 QN53=1 OR Q29w>=1 - IF ATTENDED 
INSTITUTION/PROGRAM) 


(PN: SET UP ASA FLEXIBLE GRID. ITEMS (a-e) SHOULD BE ON THE LEFT) 


50w. For each of the following activities, please indicate whether today you would feel very 
comfortable, somewhat comfortable, somewhat uncomfortable, or very uncomfortable in 
completing the activity. 


Very Comfortable 
Somewhat Comfortable 
Somewhat UNcomfortable 
Very UNcomfortable 
Don’t Know 


Blank/No Answer 


XxX O BR WO DY B 


Choosing the right courses to take 


oy 


Managing my class assignments 


9 


Managing multiple courses and other responsibilities 
d. Finding fulfilling extracurricular engagements 


e. Approaching and speaking with professors and/or instructors 
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[TOPIC: SELF CONTROL SCALE] 


(ASK EVERYONE) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-q) SHOULD BE ON THE LEFT) 


51w. For each of the following statements, select the option that best reflects you. 


Not at all like me 

A little like me 
Somewhat like me 
Mostly like me 
Very much like me 
Don’t Know 


Blank/No Answer 


x oun BF WN BR 


a. | have a hard time breaking bad habits 
b. | get distracted easily 
Cc. | say inappropriate things 
d. | refuse things that are bad for me, even if they are fun 
e. I’m good at resisting temptation 
f. People would say that | have very strong self-discipline 
g. Pleasure and fun sometimes keep me from getting work done 
h. | do things that feel good in the moment but regret later on 
i. Sometimes | can’t stop myself from doing something, even if | know it is wrong 
j. | often act without thinking through all the alternatives 
k. | organize my time and do not procrastinate (not put things off) 
I. | set appropriate priorities 
m. | practice self discipline 
n. | am able to distinguish spending for necessities versus desired purchases 
O. | set goals for myself 
p. | find ways to achieve my goals 
q. | consider possible obstacles when making plans 
(ASK EVERYONE) 
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(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-d) SHOULD BE ON THE LEFT) 


52w. For each of the following activities, select how often you do them: 


1 Never 

2 Less than once a year 

3 Once every few months 

4  Onceamonth 

5  Morethan oncea month 

8 Don’t Know 

X — Blank/No Answer 
a. Try doing new things 
b. Try anew way of acting around people 
C. Think about my future 


d. Think about who | am 


(ASK EVERYONE) 
(PN: SET UP ASA FLEXIBLE GRID. ITEMS (a-d) SHOULD BE ON THE LEFT) 


53w. For each of the following activities, select how successful you are at them: 


Not successful at all 
A little successful 
Somewhat successful 
Mostly successful 
Very successful 
Don’t Know 


Blank/No Answer 


Kx OU KR WDHB 


a. Controlling my temper 

b. Dealing with fear and anxiety 

C. Handling stress 

d. Understanding how my emotions affect how | perform 
(NEW SCREEN) 
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(ASK W1 NONRESPONDENTS) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-h) SHOULD BE ON THE LEFT) 


82. Next, please respond to the following statements. Be honest - there are no right or wrong 
answers! 


For each statement, is that very much like you, mostly like you, somewhat like you, not much 
like you, or not like you at all? 


Very much like me 
Mostly like me 
Somewhat like me 
Not much like me 
Not like me at all 


Blank/No Answer 


x oO BRB WN PR 


a. New ideas and projects sometimes distract me from previous ones 

b. Setbacks don’t discourage me 

C. | am obsessed with a certain idea or project for a short time but later lose interest 

d. lama hard worker 

e. | often set a goal but later choose to pursue a different one 

f. | have difficulty maintaining my focus on projects that take more than a few months to 
complete 

g. | finish whatever | begin 


h. lam diligent 
[TOPIC: EMPLOYMENT] 


(NEW SCREEN) 


The following are questions about your past and current employment. 


(ASK ALL) 


(PN: INSERT FIRST VERBIAGE FOR W1 RESPONDENTS; INSERT SECOND VERBIAGE FOR W1 
NONRESPONDENTS) 


W56. Since [W1 Survey Date/June 2013], have you held a paid or unpaid job, including internships? 


1 Yes 
2 No 
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X — Blank/No Answer 


(ASK IF W56=1) 


(PN: INSERT FIRST VERBIAGE FOR W1 RESPONDENTS; INSERT SECOND VERBIAGE FOR W1 
NONRESPONDENTS) 


64J. Please enter the number of jobs you have you held since [W11 Survey Date/ June 2013]: 


—___ (PN: ACCEPT 1-20) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 

(PN: ASK Q65-Q73a AND Q75-Q79 FOR EACH JOB SPECIFIED IN Q64J) 

(IF Q64J=1, INSERT FIRST VERBIAGE; IF Q64J>1 OR Q64J=8 INSERT SECOND VERBIAGE) 
(IF Q64J>1 OR Q64J=8 INSERT PARENS) 

(PN: INSERT THIRD NOTE IF TREATMENT SAMPLE) 


65. Please enter the name of the company or organization where you work, or worked: / 
Beginning with the most recent job you have held, or your current job, please enter the name 
of the company or organization: ... What is the name of the company or organization you 
worked at prior to the one you just mentioned? 


(If you are currently working at more than one job, please list the one at which you work the most hours or if 
you work equal hours at both jobs, the one where you have worked the longest.) 


(Please include any work study positions) 


(If you participated in the Urban Alliance Internship Program, please list the name of the company or 
organization where you completed your internship.) 


(Company or Organization name) 


X — Blank/No Answer 
66. Please enter your position title: 


(Position title) 


8 Don’t Know 
X — Blank/No Answer 
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67. When did you start working at (INSERT ORGANIZATION)? Please enter the month and year. 


(PN: ACCEPT 1-12/2000-2015) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
68.  Areyoustill working there? 


1 Yes 
2 No 
xX — Blank/No Answer 


(ASK Q69 IF Q68=2) 


69. Whendid you stop working there? Please enter the month and year. 


(PN: ACCEPT 1-12/2000-2015) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 


(PN: IF Q68=1, INSERT FIRST VERBIAGE IN PARENS; IF Q68=2, INSERT SECOND VERBIAGE IN 
PARENS) 


70a. (Do/did) you work a different number of hours at (INSERT ORGANIZATION) during the 
school year, than during the summer and other breaks? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q70a=1) 


(PN: IF Q68=1, INSERT FIRST VERBIAGE IN PARENS; IF Q68=2, INSERT SECOND VERBIAGE IN 
PARENS) 
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70. About how many hours (do/did) you work at (INSERT ORGANIZATION) per week during the 
school year? 


(Your best estimate is fine) 


_____ (PN: ACCEPT HOURS 0-60) 
8 Don’t Know 
X — Blank/No Answer 


(ASK IF Q70a=1) 


(PN: IF Q68=1, INSERT FIRST VERBIAGE IN PARENS; IF Q68=2, INSERT SECOND VERBIAGE IN 
PARENS) 


71. | About how many hours (do/did) you work there per week during the summer or other breaks? 
(Your best estimate is fine) 


____ (PN: ACCEPT HOURS 0-60) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK IF Q70a=2) 


(PN: IF Q68=1, INSERT FIRST VERBIAGE IN PARENS; IF Q68=2, INSERT SECOND VERBIAGE IN 
PARENS) 


72. |About how many hours (do/did) you work at (INSERT ORGANIZATION) per week? 
(Your best estimate is fine) 


1 Hours (0-60) 
X — Blank/No Answer 


(NEW SCREEN) 


(PN: IF Q68=1, INSERT FIRST VERBIAGE IN PARENS; IF Q68=2, INSERT SECOND VERBIAGE IN 
PARENS) 


73. | How (are/were) you paid at (INSERT ORGANIZATION)? 
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(Please select one of the rates listed below) 


1 Hourly 

2 ‘Daily 

3 Weekly 

4 Bi-weekly 

5 Bi-monthly 

6 Monthly 

7 Yearly 

9 The job (is/ was) unpaid 
8 Don’t Know 

Xx 


Blank/No Answer 


(NEW SCREEN) 
(ASK IF Q73=1-8) 


(PN: IF Q68=1, INSERT FIRST VERBIAGE IN PARENS; IF Q68=2, INSERT SECOND VERBIAGE IN 
PARENS) 


73a. Before taxes or other deductions, what (is/was) your (INSERT RESPONSE FROM Q73) wage 
at (INSERT ORGANIZATION) including tips and commissions? 


(Please enter your wages below) 


Hourly (1-20) 

Daily (1-150) 
Weekly (1-800) 
Bi-weekly (1-1600) 
Bi-monthly (1-1750) 
Monthly (1-3500) 
Yearly (1-60000) 
Don’t Know 


Blank/No Answer 


K ON AU KR WON B 


(NEW SCREEN) 
(ASK IF Q64=1 OR W56=1 OR #M1029 qn64=1) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-e) SHOULD BE ON THE LEFT) 
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74. — The following is a list of problems that people might have at work. Thinking about your 
current or most recent job, how often did you have trouble ...? 


Never 

Only once or a few times 
About once a week 
Almost everyday 
Everyday 

Not applicable 
Blank/No Answer 


x aA oF WN RB 


a. Getting along with your supervisor 
b. Paying attention while at work 
C. Getting along with your co-workers 
d. Dealing with customers 
e. Arriving on time for work 

(NEW SCREEN) 

(ASK IF Q68=1) 


75. | What are your usual duties or activities at this job? For example, filing, selling cars, laying 
brick, customer service. 


(Please enter response below) 


___ (Specify) 
8 Don’t Know 
X — Blank/No Answer 
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(NEW SCREEN) 
(ASK IF Q68=1) 
(PN: INSERT VERBIAGE IN PARENS IF Q68=1 FOR MORE THAN ONE JOB) 


76. 


2 0 5 


@ 


The following is a list of benefits. Are you eligible for any of the following benefits through 
your employer(s)? By eligible, we mean that the benefit is available to you now, even if you 
have decided to not receive it. 


Yes 
No 
Don’t Know 


Blank/No Answer 


Xx O©@N BR 


Health insurance 
Dental insurance 
Paid sick leave 
Paid vacation 


Employer tuition reimbursement 


(NEW SCREEN) 
(ASK IF Q68=1) 
(PN: INSERT VERBIAGE IN PARENS IF Q68=1 FOR MORE THAN ONE JOB) 


77. 


How did you find out about your current job, (that is, the one at which you work the most 
hours, or if you work equal hours at both jobs, the one where you have worked the longest)? 


(Please select only one response) 


Found on employer’s website or another website 

Saw advertisement on campus, in the community, or at a place of business 
Recommended by friends or relatives 

Recommended by Career Center or at a Job Fair 

| created the position myself or | am self-employed 

Other (SPECIFY) 

Blank/No Answer 


K Au KR WON B 
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(NEW SCREEN) 
(ASK IF Q68=1) 
(PN: INSERT VERBIAGE IN PARENS IF Q68=1 FOR MORE THAN ONE JOB) 


78. Was there someone who suggested that you apply for your current job(s) or helped you get 
the job(s), other than the person who hired you? Do not include references requested by the 


employer. 

1 Yes 

2 No 

X — Blank/No Answer 


(ASK IF Q78=1) 
(PN: INSERT VERBIAGE IN PARENS IF Q68=1 FOR MORE THAN ONE JOB) 
(INSERT RESPONSES 5, 6, 7 IF TREATMENT GROUP) 


79. What was that person’s (or persons’) relationship to you? 


(Please select all that apply) 


1 
2 
3 
4 
5 
6 
7 
8 
9 


Parent or Foster Parent 

Relative 

Friend or Acquaintance 

An employer or co-worker (includes current and former) 
Urban Alliance Program Coordinator (“PC”) 

Urban Alliance Alumni Services Staff 

Other Urban Alliance Staff 

Caseworker 


Teacher 


10 Clergyperson 
11 Other 
99 Refused 


(NEW SCREEN) 
(ASK IF W56=2) 


(PN: INSERT FIRST VERBIAGE FOR W1 RESPONDENTS; INSERT SECOND VERBIAGE FOR W1 
NONRESPONDENTS) 


80. Have you looked for a job since [W1 Survey Date/June 2013]? 
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1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(qn68_01=2 OR 56_w2=2) 


81. What is the main reason you are not working? 


(Please select only one response) 


Going to school 

Cannot find work 

No need or no desire 

Taking care of home or family 

Previous work was temporary, seasonal, or completed 
Laid off 

Quit 

Fired 

Moved 


So ON WO WN FPF W DN BF 


BR 
oO 


Incarcerated 


a= 
= 


Temporarily disabled and unable to work 


BE 
Nh 


Permanently disabled and unable to work 


as 
wo 


Changing jobs 
Couldn't afford or find childcare 


Be 
ms 


15 Transportation issues or long distance 


16 Not enough skills 
17 Other (SPECIFY) 
X — Blank/No Answer 


NEW WAVE 2 
[TOPIC: URBAN ALLIANCE ALUMNI SERVICES] 


(ASK Q77-Q83 OF TREATMENT GROUP ONLY) 
READ: The following questions are about Urban Alliance’s Alumni Services. 
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W77. Have you ever been in contact with Alumni Services or other staff at Urban Alliance since 


September 2013? 

1 Yes 

2 No 

8 Don’t Know 

X — Blank/No Answer 


(ASK IF Q77 = 


1) 


W78. Since September 2013, who initiated your contact with Urban Alliance staff? 


Xx OM@ WN KB 


(ASK IF Q77 = 


| reached out to Urban Alliance staff 

Urban Alliance staff reached out to me 

| reached out to Urban Alliance staff and Urban Alliance staff reached out to me 
Don’t know 


Blank/No Answer 


1) 


W79. Why have you been in contact with Alumni Services or other staff at Urban Alliance since 
September 2013? Please select all that apply. 


CO NWO WN FF W NY BF 


(ASK IF Q77 = 


A career planning or job search issue 

Aworkplace issue 

An education planning or financial aid issue 

An issue | was experiencing in school 

A personal issue 

Speaking with current Urban Alliance students or attending other Urban Alliance events 
To share updates about my career, education, or other life events 

Other (Specify) 

Don’t Know 


Blank/No Answer 


1) 


W80. How often have you interacted with Urban Alliance staff or attended Urban Alliance events 
since September 2013? 
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Less than once a year 

1 to 5 times a year 

6 to 12 times a year 

More than 12 times a year 


Blank/No Answer 


x BR ODN B 


(ASK IF Q77 = 1) 
(PN: SET UP ASA FLEXIBLE GRID. ITEMS (a-d) SHOULD BE ON THE LEFT) 


W681. 


So 


Since September 2013, have you attended any of the following as an alumni: 


Yes 
No 
Don’t Know 


Blank/No Answer 


xXx ONY & 


Career Day 
Field trip 
Alumni panel and networking session 


Other Urban Alliance events 


(ASK IF Q77 = 1) 


W82. Did Urban Alliance connect you with a summer internship for college students? 


Yes 
No 
Don’t Know 


Blank/No Answer 


Xx O©N RB 


(ASK IF Q77 = 1) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-f) SHOULD BE ON THE LEFT) 


W83. To what extent do you agree or disagree with the following statements? Since September 


2013, Urban Alliance staff have been: 


1 Strongly disagree 
2 Disagree 
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Neither agree nor disagree 
Agree 

Strongly agree 

Don’t know/Not applicable 
Blank/No Answer 


x on fF W 


a. a good resource for education, career, and general life guidance 

b. helpful in advancing my career 

C. helpful in dealing with workplace issues 

d. helpful in navigating my education options 

e. helpful in finding and pursuing options to pay for education 
helpful in dealing with issues I’ve experienced in school 

g. helpful in dealing with personal or family issues 


(ASK ALL TREATMENT GROUP) 


Wé84. To what extent do you agree or disagree with this statement: It has been easy to contact 
Urban Alliance staff after participating in the program. 


Strongly disagree 

Disagree 

Neither agree nor disagree 
Agree 

Strongly agree 

| haven't tried to contact them 


Blank/No Answer 


x aun fF WY BR 


[TOPIC: ASSETS AND SAVINGS] 


(NEW SCREEN) 


This next set of questions asks you about your assets and savings. 


(ASK ALL) 


83. Doyou have achecking account? 


1 Yes 
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2 No 
X — Blank/No Answer 


(ASK IF Q83=1) 


84. What is your approximate current balance in your checking account? 


(1-50000) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


85. Doyouhave a savings account? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q85=1) 


86. What is your approximate current balance in your savings account? 


(1-50000) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


87. Doyouhave any other types of accounts where you have money available to you? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q87=1) 
(PN: ALLOW FOR MULTIPLE RESPONSES) 
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88. What kind of accounts do you have? 


Specify accounts 1 


Specify accounts 3 


( ) 
(Specify accounts 2) 
( ) 
( ) 


Specify accounts 4 


8 Don’t Know 
X — Blank/No Answer 


(ASK IF Q87=1) 
(PN: IF RENTERED MULTIPLE ACCOUNTS IN Q88, INSERT SECOND VERBIAGE IN PARENS) 


89. Whatis your approximate total current balance in (this / these) accounts? 


(1-50000) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


90. Doyou own any vehicles such as a car, van, truck, jeep, or motorcycle? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q90=1) 


91. Altogether, how much could you sell these vehicles for? 
(Your best estimate is fine) 
(1-50000) 
8 Don’t Know 


X — Blank/No Answer 


(NEW SCREEN) 
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(ASK ALL) 


92. Have you taken out loans to help pay for college or other programs? 


Yes 
No 
Don’t Know 


Blank/No Answer 


Xx O©N BR 


(ASK IF Q92=1) 


93. | What is the total dollar amount you have taken out in loans to pay for college or other 
programs? 


(1-100000000) 
8 Don’t Know 
X — Blank/No Answer 


[TOPIC: FAMILY AND HOUSING] 


(Q94 - Q97 ASKED IF W1 NONRESPONDENTS) 


(NEW SCREEN) 


The following are questions about your family and housing situation. 


(ASK W1 NONRESPONDENTS) 


94. Please select your father’s highest level of education: 


(Please select only one response) 


Did not complete high school 

High school graduate (or equivalent) 
Some college (1-4 years, no degree) 
Associate’s degree 

Bachelor’s degree 


Master’s degree or higher 


N BD oO BP WY KB 


Not applicable 
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8 Don’t Know 
X — Blank/No Answer 
(NEW SCREEN) 


(ASK W1 NONRESPONDENTS) 


95. Please select your mother’s highest level of education: 


(Please select only one response) 


K ON AU KR WN BE 


Did not complete high school 

High school graduate (or equivalent) 
Some college (1-4 years, no degree) 
Associate’s degree 

Bachelor’s degree 

Master’s degree or higher 

Not applicable 

Don’t Know 


Blank/No Answer 


(NEW SCREEN) 
(ASK W1 NONRESPONDENTS) 


96. _ Have any of your brothers or sisters gone to college? 


Xx OM@ WN KB 


Yes 

No 

Not applicable (Do not have any brothers or sisters) 
Don’t Know 


Blank/No Answer 


(ASK W1 NONRESPONDENTS) 


97. Have any other family members that you’re close to gone to college? 


1 
2 
3 


Yes 
No 
Not applicable 
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8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


98. Arethere any children who are living with you and in your care? 


“In your care” means that you are legally responsible for the child or have formal custody for the 
child. Informal care arrangements, such as taking care of a sister’s child while she is at work, should 
not be included. 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q98=1) 


99. Please enter the number of children that currently live with you and are in your care: 


(1-10) 
X — Blank/No Answer 


(NEW SCREEN) 

(ASK IF Q99=1) 

(PN: ASK Q100-Q102 FOR EACH CHILD SPECIFIED IN Q99) 

(IF Q99=1, INSERT FIRST VERBIAGE; IF Q99>1 OR Q99=8 INSERT SECOND VERBIAGE) 


100. Please enter his or her first name: / Please enter the first name of your oldest child: ... Please 
enter the name of your next oldest child: 


(First name) 


X — Blank/No Answer 


(ASK IF Q98=1) 
101. What is his or her birthday? 


(PN: ACCEPT 1-12/1-31/2005-2015) 
8 Don’t Know 
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X — Blank/No Answer 


(ASK IF Q98=1) 
102. Do you receive child support for this child? 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


106. Do you have biological children that do not live with you? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q106=1) 
107. Are you required by court order to pay child support for these children? 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


108. Are you or your partner expecting a child? 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


109. What is your current marital status? Are you... 
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(Please select only one) 


1 Single, never married 
2 Living with partner 
3 Married 
4  Separated/divorced/widowed 
X — Blank/No Answer 

(NEW 2014) 

(NEW SCREEN) 

(ASK IF W1 NONRESPONDENT) 

2. Before you turned 18, was there ever a period of four months or more when you did not live 


with at least one of your biological or adoptive parents? 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW 2014) 
(ASK IF Q2=1) 


2a. Select the ages below when you did not live with at least one of your biological or adoptive 
parents for four months or more. 


(Please select all that apply) 


Before your 1st birthday 
1 year old 

2 years old 

3 years old 


5 years old 


6 years old 


1 

2 

3 

4 

5 Ayears old 
6 

7 

8  7yearsold 
9 


8 years old 
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10 9years old 
11 10years old 
12 11 years old 
13 12years old 
14 13 years old 
15 14years old 
16 15years old 
17 16years old 
18 17years old 
19 18 years old 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK IF Q2=1) 


2b. Whenyou were not living with either of your biological or adoptive parents, with whom or 
where did you live? 


(Please select all that apply) 


With relatives who were also my foster parents 
With relatives who were not my foster parents 
With my foster parent(s) who are unrelated to me 
With a friend's family (not foster care) 

A group home or residential facility 

On my own (alone) 

Shared housing with a friend or roommate 


With my spouse, partner, or boyfriend, or girlfriend 


So ON WO WN FP W NY BF 


At a homeless shelter or emergency housing 


as 
(oe) 


Homeless 


a= 
= 


College dormitory, fraternity, sorority 
12 Transitional housing 

13 Jailor prison 

14 Job Corps 

15 Other (SPECIFY) 

X — Blank/No Answer 
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(NEW 2014) 
(ASK IF Q2B=1, 2) 
(PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-d) SHOULD BE ON THE LEFT) 


3. Were the relatives...? 
1 Yes 
2 No 


X — Blank/No Answer 


a. Grandparents 
b. Aunts or Uncles 
C. Older brothers or sisters 


d. Someone else (SPECIFY) 


(NEW SCREEN) 
(ASK ALL) 


110. What best describes your current living situation? Are you... 


(Please select only one response) 


In student housing 

In military housing 

Living with parents 

Living with other adult family member or guardian 
Living with significant other 

Living with roommates in non-student housing 
Living alone 


Homeless or living in a shelter 


Oo ON WO WN FPF WN KB 


Incarcerated 
10 Other (Specify) 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 
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111. Please enter your current address: 


Your current address does not have to be your permanent address but should be where you 
currently reside. It can be a school address. 


STREET ADDRESS) 
CITY) 

STATE) 

ZIPCODE) 
Blank/No Answer 


( 
( 
( 
( 


x BR ODN B 


112. Have you been living there since [W1 Survey Date/June 2013]? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q112=2) 

(PN: ALLOW BOTH MONTH AND YEAR) 

(PN: ONLY ALLOW JUNE 2013TO PRESENT DATE) 
113. When did you move there? 


(PN: ACCEPT 1-12/2013-2015) 
8 Don’t Know 
X — Blank/No Answer 


(ASK IF Q112=2) 


114. What was the main reason that you moved there? 


(Please select only one) 


School (Going to college, leaving college, or wanting an easier commute to school) 
Work (Getting a new job, losing a previous job, or wanting an easier commute to work) 


Money (Wanting a cheaper place, not having enough money for rent) 


B WO DD BB 


Legal problems (Being arrested or incarcerated) 
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5 Your health 
6 Wanting to live with someone different 
7 Wanting to be on your own 
8 Needing to help a family member 
9 Wanting to live in a better neighborhood 
10 Needing to find something more permanent 
11 Other reason (SPECIFY) 
X — Blank/No Answer 
(NEW SCREEN) 


(ASK IF Q112=2) 


115. Please enter the address were you living before this last move: 


x BR ODN B 


MAILING ADDRESS) 
CITY) 

STATE) 

ZIPCODE) 

Blank/No Answer 


( 
( 
( 
( 


(ASK IF Q112=2) 
(PN: ALLOW BOTH MONTH AND YEAR) 
116. Whendid you move there? 


1 
2 
8 
X 


(PN: ACCEPT 1-12/1994-2015) 
Lived there since birth (did not move there) 
Don’t Know 


Blank/No Answer 


(ASK IF Q116=1 AND Q116>6/1/13) 


117. What was the main reason that you moved there? 


(Please select only one) 
1 School (Going to college, leaving college, or wanting an easier commute to school) 


2 


Work (Getting a new job, losing a previous job, or wanting an easier commute to work) 
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Money (Wanting a cheaper place, not having enough money for rent) 
Legal problems (Being arrested or incarcerated) 


Your health 


3 
4 
5 
6 Wanting to live with someone different 
7 Wanting to be on your own 

8 Needing to help a family member 

9 Wanting to live in a better neighborhood 

10 Needing to find something more permanent 
12 Lived there since birth (did not move there) 
11 Other reasons (SPECIFY) 


X — Blank/No Answer 
(PN: IF Q116>6/1/12, ASK Q115-117 UNTIL Q116<6/1/12, UP TO FIVE TIMES) 
[TOPIC: HEALTH] 


(NEW SCREEN) 


The following questions are about your health. 


(ASK ALL) 


118. In general, would you say your healthis... 


Excellent 
Very good 
Good 
Fair 

Poor 


Blank/No Answer 


x OW BR WOW NY BR 


(NEW SCREEN) 
(ASK ALL) 


119. Inthe past 12 months, have you delayed getting medical or dental care for any reason when 
you really needed it? 


1 Yes 
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2 No 
X — Blank/No Answer 


(ASK IF Q119=1) 


120. What was the main reason you delayed getting care? 


(Please select only one response) 


Didn’t know whom to go see 

Had no transportation 

No one available to go along 

Parent or guardian would not go 

Didn’t want parents to know 

Difficult to make appointment 

Afraid of what the doctor would say or do 


Thought the problem would go away 


So ON WO WN FPF WN BF 


Couldn’t pay 
10 Other (Specify) 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


Q121.Do you have health insurance? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q121=1) 
(PN: INSERT VERBIAGE IN FIRST PARENS AND ITEM 3 IF Q109=3, 4) 


122. What is the source of your health insurance? Would you say it’s through a parent or guardian, 
your college, (your spouse,) the government (like Medicaid), your employer, or something 
else? 


1. Parent/Guardian 
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College 

Spouse 

Government (Medicaid) 
Employer 

Other (SPECIFY) 

Don’t Know 


Blank/No Answer 


x AOA HO KF W DN 


(ASK IF Q121=2) 


123. Why are you not covered by health insurance? 


(Please select only one) 


Too expensive 

No longer eligible 

Healthy or don’t need insurance 

Too much hassle to stay enrolled 

Did not submit paperwork/ documents or pay premiums 
Doctors would not accept insurance 

Gap in coverage changing plans 

Other (SPECIFY) 

Blank/No Answer 


K ON AU KR WON BE 


(NEW SCREEN) 


(PN: INSERT FIRST VERBIAGE FOR W1 RESPONDENTS; INSERT SECOND VERBIAGE FOR W1 
NONRESPONDENTS) 


The next set of questions is about things that have happened to you since [W1 Survey Date/June 
2013]. The following questions pertain to only your personal situation, not that of other family 
members. These questions focus on hardships that many people experience at one time or another. You 
can choose to skip any questions at any time without penalty. 


[ERROR MESSAGE 2] 


(PN: IF ARESPONDENT FAILS TO PROVIDE A RESPONSE TO A CERTAIN QUESTION WITHIN THE 
SERIES Q124-Q146, PLEASE INSERT EM2. THE ERROR MESSAGE SHOULD APPEAR ABOVE THE 
QUESTION MISSED (ON THE SAME SCREEN) IN BOLD BLACK TEXT.) 
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EM2 Please remember all your responses will be kept confidential. Your answers are very 
important to us. Can you please take a moment to respond to the question below? 


If you have any questions about the study, please call Kasey Meehan at 484-840-4399. 


(ASK ALL) 


124. Since [W1 SURVEY DATE/June 2013], have you received any cash assistance, welfare, or 
emergency help from a state or county welfare program, such as TANF, for a month or more? 


(Your responses should be based only on your personal situation, not that of other family members) 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK ALL) 
125. Since [W1 SURVEY DATE/June 2013], have you received Food Stamps? 


(Your responses should be based only on your personal situation, not that of other family members) 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK ALL) 


126. Since [W1 SURVEY DATE/June 2013], have you received any governmental housing 
assistance in paying rent, such as through public housing or Section 8? 


(Your responses should be based only on your personal situation, not that of other family members) 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 
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127. Still thinking about things that have happened since [W1 SURVEY DATE/June 2013], have 
you... had to sleep outside or in a shelter on any night? 


1 Yes 
2 No 
X — Blank/No Answer 


(Ask if Q127=1) 


128. How frequently have you slept in a shelter since [W1 SURVEY DATE/June 2013]? Would you 
Say... 


Once or twice 
About once a month 
A few times a month 
About once a week 
A few times a week 
Most days 

Every day 

Blank/No Answer 


Xx NWO WO FPF W DN BF 


(NEW SCREEN) 
(ASK ALL) 


129. Still thinking about things that have happened since [W1 SURVEY DATE/June 2013], has 
someone you're close to experienced a major illness or disabling condition? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK ALL) 


130. Still thinking about things that have happened since [W1 SURVEY DATE/June 2013], has 
someone you're close to been incarcerated? 


1 Yes 


2 No 
X — Blank/No Answer 


APPENDIX G ize 


(ASK ALL) 


131. Still thinking about things that have happened since [W1 SURVEY DATE/June 2013], has 
someone you're close to passed away? 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


132. Still thinking about things that have happened since [W1 SURVEY DATE/June 2013], have 
you had to cut the size of your meals or skip meals because there wasn’t enough money for 


food? 
1 Yes 
2 No 


X — Blank/No Answer 


(NEW SCREEN) 


In answering this next set of questions, think about your experience with illegal or harmful activities. 
Please remember this survey is confidential and your answers will be kept completely private. 


(ASK ALL) 


133. Have you ever been arrested? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q133=1) 


134. Howold were you the first time you were arrested? 
(10-20) 


8 Don’t Know 
X — Blank/No Answer 
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(ASK IF Q133=1) 
135a. Have you been arrested for a felony since [W1 SURVEY DATE/June 2013]? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q133=1) 
135b. Have you been arrested for a misdemeanor since [W1 SURVEY DATE/June 2013]? 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK IF Q133=1) 
136. Since [W1 SURVEY DATE/June 2013], have you ever been convicted of a crime as an adult? 


1 Yes 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 


138. Have you ever had at least one drink of alcohol (as in, more than a few sips)? 
1 Yes 
2 No 


X — Blank/No Answer 


(ASK IF Q138=1) 
139. During the past 30 days, on how many days did you have at least one drink of alcohol? 


(Your best estimate is fine) 
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O days 

1or 2 days 

3 to 5 days 

6 to 9 days 
10 to 19 days 
20 to 29 days 
All 30 days 
Don’t Know 


Blank/No Answer 


KX ON AWN KR WHNY B 


(ASK IF Q139=1-7) 


140. During the past 30 days, on how many days did you have 5 or more drinks of alcohol ina row, 
that is, within a couple of hours? 


(Your best estimate is fine) 


O days 

1 day 

2 days 

3 to 5 days 

6 to 9 days 

10 to 19 days 
20 or more days 
Don’t Know 


Blank/No Answer 


K ON AU BR WN BE 


(ASK IF Q138=1) 


141. Howold were you when you had your first drink of alcohol, other than a few sips? 
(10-22) 
8 Don’t Know 


X — Blank/No Answer 


(NEW SCREEN) 
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(ASK ALL) 


142. Have you ever used marijuana? 


1 Yes 
2 No 
X — Blank/No Answer 


(ASK IF Q142=1) 


143. How many times in the past 30 days did you use marijuana? 


(Your best estimate is fine) 


O times 

1or 2 times 

3 to 9 times 

10 to 19 times 
20 to 39 times 
40 or more times 
Don’t Know 


Blank/No Answer 


x AA HT FF W DN BF 


(ASK IF Q142=1) 


144. Howold were you when you tried marijuana for the first time? 


(10-22) 
8 Don’t Know 
X — Blank/No Answer 


(NEW SCREEN) 
(ASK ALL) 
[PN: SET UP ASA FLEXIBLE GRID. ITEMS (a-f) SHOULD BE ON THE LEFT.] 


145. Have you ever... 


1 Yes 
2 No 
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8 Don’t Know 
X — Blank/No Answer 


a. Used any form of cocaine 

b. Sniffed glue, breathed aerosol, or inhaled paints or sprays to get high 

ce Used heroin 

d. Taken methamphetamines 

e. Taken ecstasy 

if. Taken pills or shots without a doctor’s prescription to get high 
(NEW SCREEN) 


(ASK Q146 FOR EVERY ITEM A-F IN Q145=1) 
[PN: SET UP AS A FLEXIBLE GRID. ITEMS (a-f) SHOULD BE ON THE LEFT.] 


146. 


ao 9 fF Y 


@ 


In the past 30 days, how many times have you... 


(Your best estimate is fine) 


O times 

1or 2 times 

3 to 9 times 

10 to 19 times 
20 to 39 times 
40 or more times 
Don’t Know 


Blank/No Answer 


x AA OT FF W DN BF 


Used any form of cocaine 

Sniffed glue, breathed aerosol, or inhaled paints or sprays to get high 
Used heroin 

Taken methamphetamines 

Taken ecstasy 


Taken pills or shots without a doctor’s prescription to get high 


(NEW SCREEN) 
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Thank you, your responses are very helpful! 
We need to confirm your contact information to send you your $40 gift card. 


(ASK EVERYONE) 


We want to verify that we have your correct name and birthday. 
2. Please confirm your first name: (INSERT FIRST NAME) 


1 Name is correct 

2  Nameis correct but spelled incorrectly. Please enter the correct spelling (Specify) 
3  Nameisincorrect. Please enter your first name: (Specify) 
Xx 


Blank/No Answer 
2L. Please confirm your last name: (INSERT LAST NAME) 


1 Name is correct 

2  Nameis correct but spelled incorrectly. Please enter the correct spelling (Specify) 
3  Nameisincorrect. Please enter your first name: (Specify) 
Xx 


Blank/No Answer 
(PN: IF Q1 AND Q2=3, INSERT TERM2) 


(ASK EVERYONE) 
3. Please confirm your birthday: (INSERT BIRTHDAY) 


1 Birthday is correct 

2 _ Birthday is incorrect. Please enter the correct birthday: 
(PN: ACCEPT 1-12/1-31/1993-2000) 

X — Blank/No Answer 


(ASK ALL) 
147. Please confirm your current mailing address (INSERT ADDRESS FROM Q111): 


1. Yes, information is correct 
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2 No, information is incorrect (PN: MAILING ADDRESS, CITY, STATE, ZIPCODE) 
X — Blank/No Answer 


(ASK ALL) 
(PN: ALLOW 10 DIGIT PHONE NUMBER) 


148. Please enter your home phone number: 


___ (Specify) 
2 No 
X — Blank/No Answer 


(ASK ALL) 
(PN: ALLOW 10 DIGIT PHONE NUMBER) 


149. Please enter your cell phone number: 


___ (Specify) 
2 No 
X — Blank/No Answer 


(ASK ALL) 


150. Can you receive text messages? 


Yes 
No 
Don’t Know 


Blank/No Answer 


x ONY BR 


(ASK ALL) 


151. Please enter your email address? 


___ (Specify) 
2 No 
X — Blank/No Answer 


(ASK IF Q151=1) 
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152. Please enter an alternate email address: 


_____ (Specify) 

2 No 

8 Don’t Know 

X — Blank/No Answer 


(NEW SCREEN) 
(ASK ONLY IF SSN NOT ON FILE) 
(PN: ALLOW 9 DIGIT PHONE NUMBER)\ 


165. Wedon't have your Social Security Number on file. Can you please provide it? Again, your 
information will be kept completely confidential and will only be accessible to people on our 
research team. 


(PN: ACCEPT xxx-xx-xxxx) 
2 No 
X — Blank/No Answer 


(NEW SCREEN) 


Those are all the questions | have for you today. Thank you very much for your time and cooperation. 
We will be sending a $40 gift card to the mailing address you provided. 


If you have questions about the study, please contact the study’s director, Kasey Meehan at SSRS at 
484-840-4399. 
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Notes 


10. 


11. 
12. 


13. 


In our interim report (Theodos, Pergamit, Hanson, et al. 2016), we did not include a site variable (Washington, 
DC, or Baltimore) in the regression analysis; however, this measure was included in the analysis for this report. 
Inclusion of a site variable resulted in only one material change: the coefficient on taking the ACT for the full 
group is no longer statistically significant. 


US Census Bureau, 2012 American Community Survey, one-year estimates. 


Estimates are for the graduating class of 2013. Baltimore estimates are from 2013 Maryland Report Card, 
“Baltimore City Graduation Rate: Four-Year Adjusted Cohort,” Maryland State Department of Education, 
accessed May 20, 2014, 
http://www.mdreportcard.org/CohortGradRate.aspx?PV=160:12:30:XXXX:1:N:0:13:1:1:0:1:1:1:3. DC 
estimates are from “DC 2013 Adjusted Cohort Graduation Rate,” District of Columbia Office of the State 
Superintendent of Education, accessed May 20, 2014, 
http://osse.dc.gov/sites/default/files/dc/sites/osse/publication/attachments/DC%202013%20ADJUSTED%2 
OCOHORT%20GRADUATION%20RATE%20state%20summary_O.pdf. The DC percentage does not include 
DC public charter schools. 


Current Population Survey, “School Enrollment,” table 1, Enrollment Status of the Population 3 Years Old and 
Over, by Sex, Age, Race, Hispanic Origin, Foreign Born, and Foreign-Born Parentage: October 2014. 


National Center for Education Statistics, 2014 Digest of Education Statistics, table 302.30, Percentage of 
Recent High School Completers Enrolled in 2-Year and 4-Year Colleges, by Income Level: 1975 through 2013. 


National Center for Education Statistics, 2014 Digest of Education Statistics, table 326.10, Graduation Rates 
of First-Time, Full-Time Bachelor's Degree-Seeking Students at 4-Year Postsecondary Institutions, by 
Race/Ethnicity, Time to Completion, Sex, and Control of Institution: Selected Cohort Entry Years, 1996 
through 2006. 


2014 American Community Survey 1-year estimates taken from “Median Earnings in the Past 12 Months (in 
2014 Inflation-Adjusted Dollars) by Sex by Educational Attainment for the Population 25 Years and Over.” 


College Board, 2009, “Lifetime Earnings by Education Level,” figure 1.2, Expected Lifetime Earnings Relative to 
High School Graduates, by Education Level. 


Current Population Survey Household Data 2014, table 3, Employment Status of the Civilian Noninstitutional 
Population by Age, Sex, and Race; and table 4, Employment Status of the Hispanic or Latino Population by Age 
and Sex. 

“Youth Apprenticeship Program,” Georgia Department of Education, 2015. 
https://www.gadoe.org/Curriculum-Instruction-and-Assessment/CTAE/Pages/Y outh-Apprenticeship- 
Program.aspx;. “Youth Apprenticeship Program Information,” Department of Workforce Development, State 
of Wisconsin, accessed January 4, 2017, https://dwd.wisconsin.gov/youthapprenticeship/program_info.htm 


The GED consists of four tests that certify passers’ high school-level academic skills. 


For Baltimore, we used the “2011 HSA English Data” and “2011 HSA Algebra Data’ data files from “Data 
Downloads,” 2013 Maryland State Report Card, accessed June 17, 2013, 
http://msp.msde.state.md.us/downloadindex.aspx?K=99AAAA. For DC, we used data reports for each school 
from 2011, accessed through “Assessment and Accountability in the District of Columbia,” District of 
Columbia Office of the State Superintendent of Education, accessed June 17, 2013, 
http://nclb.osse.dc.gov/reportcards.asp. 


National Center for Education Statistics, “Public Elementary/Secondary School Universe Survey Data, 2009- 
10,” accessed June 6, 2013, http://nces.ed.gov/ccd/pubschuniv.asp. 
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14. 


15. 


16. 


17. 


18. 


19. 


The random effects model modifies the earlier regression framework to 
ysi* = B1indsi + B2neighsi + as + esi 


such that as is the high school-specific effect on ysi. In arandom effects model, the assumption is that the high 
schools included in the analysis are a subset of a greater pool of high schools, such that as is distributed 
normally with mean zero and variance oa2. This assumption allows the regression to use both between and 
within variation in the data. 


In our interim report (Theodos, Pergamit, Hanson, et al. 2016) we did not include DC in the regression analysis. 
This changed only one primary outcome in the final report. The coefficient on taking the ACT is no longer 
significant for the full group. 


In this report, African American refers to non-Hispanic African American, white to non-Hispanic white, and 
other to non-Hispanic other. 


There are exceptions; some programs appear to be targeted toward (or more attractive to) males. For example, 
the National Guard Youth ChalleNGe Program is 88 percent male (Millenky et al. 2011), and Job Corps is 59 
percent male (Schochet, Burghardt, and McConnell 2006). 


In recent years, many Baltimore and Washington, DC, schools have become certified to offer free lunch to all 
students. Therefore, eligibility of individual students is no longer determined; the statistics shown here give an 
estimate of what eligibility would be in these schools if it were still determined at the student level. In the 
2012-13 school year, students from a four-member household with income below $42,643 qualified for a 
reduced-price meal, according to federal guidelines. 


Of the 965 youth for whom data were available on GPA as of junior year, school records provided information 
for 657 youth (68 percent). For 253 youth (26 percent), this information was provided by a school counselor. 
Finally, 55 youth (6 percent) provided this information directly. 
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